Category Archives: Blog

Cost of WordPress blog running on Google AppEngine

Having recently migrated all of my blog/site content to Google AppEngine away from my prior $10/month hosting I was curious about what the monthly cost of running the site would be. My blog went live on AppEngine on Nov 13th and with the default instance settings was running ~$2.00/day at roughly 50-55 instance hours the vast majority of which is servicing requests from crawlers.

Tuning Google AppEngine, For Cost

Idle Instances

As mentioned above I started with the default settings and the first significant change was to set Idle Instance Max to 1 as running multiple instances chews through the free 28 hours at an xN multiplier rate. This change alone had a significant impact on reducing overall instance hours though clearly impacts the performance of my blog. I’ve tested the performance and for a blog I feel it’s suitable so I’m comfortable leaving things at this level for awhile.

robots.txt and dos.yaml

I’ve been running another site on AppEngine which has both significantly more pages and traffic than my blog though I’ve been able to optimize it to stay under 28 instance hours and one key factor has monitoring bot traffic and blocking via either robots.txt or dos.yaml.

Google Cloud SQL Costs

So far for the month of November which is missing roughly 12.5 days worth of actual traffic to my blog I’ll be around $15. In addition to normal site traffic (crawlers & users) I’m currently running JetPack which appears to ping back to my domain for tracking stats etc. I’ll need to evaluate the possible benefits of using dedicated memcache over the Cloud SQL expense as most of the site’s content is static. I’ve configured WordPress as recommended by Google using Memcached plugin as well as Batcache.

SSL Administration

I’ll admit I mistakenly hadn’t thought too much about SSL prior to finally switching my domain over to Google’s servers. At that point, I faced the issue of having to buy SSL support for AppEngine which would certainly add to the monthly cost of running my blog. Seeing as how I’m the only administrator this wasn’t exactly an attractive option given the traffic challenges my blog has faced coupled with the desire to keep costs down. I’ve instead opted to use the free SSL on the appspot.com version of my site to manage WordPress administration which works *fairly* well though there are places (like the Themes Editor) which expect to load all content from the hosting domain which in my case doesn’t work. I suspect there are other issues here as well that I simply haven’t run into yet but time will certainly reveal those issues too.

At this point, I’m fairly happy with the switch as WordPress is far more advanced that BlogEngine.NET 2.X not to mention it has clients for iPad and iPhone which is likely where I’ll be posted from the vast majority of the time. That said, I’ll continue watching the hosting costs and learning more about the SSL issues.

Updated Results (Dec 6, 2013)

I’ve been running with the above modifications since Nov. 21, 2013 and my blog is running about 24-26 instance hours per day, within Google’s free 28 hours. It looks like my associated Cloud SQL instance is running about $14-16/month which is roughly comparable to where I was hosting before. Prior to these changes my instance cost was nearing $2/day and typically exceeding 50 instance hours.

WordPress on Google AppEngine and Migrating from BlogEngine.NET

In 2004 I started blogging on my own domain using discountasp.net as they offered a competitive hosting package based on Microsoft technology. Fast forward nearly a decade and I haven’t used Microsoft technology for nearly 5 years and maintaining my Blogengine.NET blog had become an increasing challenge. Upgrades were very difficult and I found myself stuck on a tweaked version that couldn’t have easily be upgraded. Recently, Google added support for PHP on Google Appengine and posted steps to get WordPress running so I applied for and got early access and I’ve been slowing migrating all of my content.

I’m writing this post prior to completing the migration as the number of steps involved has been growing and I figured it might be helpful to track what was required so I don’t forget and to share what I found. The list here is not exhaustive by any stretch and it’s likely written in such a way for me to remember than perhaps general utility though nonetheless here it is.

Once I got a basic install working which was rather straight forward I spent several days getting reacquainted with WordPress which I’d used many years go though only locally. Needless, to say it’s come a long way and with an iOS client on both the iPhone and iPad it looks really promising.

WordPress Plugins

  • Memcached plugin (and no this doesn’t activate!)
  • BlogML pluggin
  • Redirection plugin
  • Creative Commons
  • Google Analytics for WordPress
  • Adsense Explosion
  • Pushover Notifications

Importing My Blog Content

First was the task of researching tools to import BlogEngine.NET’s BlogML format for which I found a plugin. The plugin wasn’t just drop in and go and I ran into issues with it processing my BlogML.xml file which required manually editing the XML as well as finding and patching XPath.class.php to v1.148 as it was outdated in the sources I’d found.

I also had to patch function parse_blogml in blogml-importer.php to read my BlogML.xml file from my WordPress dedicated Google Cloud Storage bucket like this (and yes, figuring out this URL took a fair bit of Googling where wpsdt-blog-files refers to my Google Cloud Storage bucket):

$sXml = file_get_contents('gs://wpsdt-blog-files/BlogML.xml');

Next I found my BlogML.xml file was too large so I had to chunk it down into several different smaller files in order for the import to proceed a process which took a long time as AppEngine repeatedly timed out importing the .xml file though it eventually worked. Oh yes, it took quite a bit of trial and error to figure out the various file permissions that had to be granted to get access from a WordPress plugin to Google Cloud Storage including the fact that I found myself cursing Google’s new Cloud Console with buttons to change permissions that only appear once you check the bucket you want to change. Perhaps needless to say it would have been awful nice if they were just disabled so you at least you knew they existed.

Setting up Redirects

I exported BlogEngine.NET to BlogML.xml and extracted all post URL’s to build a CSV mapping to import into the Redirect plugin as WordPress and BlogEngine.NET don’t have compatible slugs. (I’m still manually correcting the dash “-” placement in over 600 slugs as no simple pattern would have worked).

Setup redirect for BlogEngine.NET dynamic image serving /blog/image.asx?picture=(.*) -> /AppData/files/\1

Setup redirect for blog archives /blog/Archives.aspx -> /blog/2013 (close enough)

Redirect BlogEngine.NET syndication link /blog/SyndicationService.asmx/GetRss -> /blog/feed

Fix up FeedBurner RSS source link /blog/feed

Set Site Address (URL) to /blog so I could continue to use appspot.com’s support for SSL for site administration without paying for an SSL cert on www.stevetrefethen.com and saving myself the cost of the cert and what looks like $9/month for SSL support on AppEngine.

Of course, I learned a hard lesson along the way by changing both the WordPress Address and Site Address URL’s in such a way that created an infinite redirect loop. To fix this I had to install Google’s CLI tools for accessing MySQL on Google Cloud SQL and manually update the wp_options table to correct my mistake.

School Blog Content

While I was working on the the Scotts Valley Parcel Tax effort I setup a secondary BlogEngine.NET install (before it supported multiple blogs) and all of that content had to be exported, imported, and redirected as well so repeat many of the above steps.

Wiki Content

I was a fan of ScrewTurn wiki and had been running a few installs of that as well so I moved a number of those pages here manually largely but cut/paste as there is/was not export/import available.

Handling Static Image Files

Over the years I’ve had various image files scattered around my site thus I had to copy these paths from my old site into the new WordPress install which included:

  • /files
  • /blog/AppData/files
  • /blog/images

Upload various media files to my WordPress Google Cloud Storage bucket.

Domain Name Transfer

I’d not done a domain name transfer before so this was my first such experience and while it all went smoothly the only problem was a very poorly written email from OpenSRS (whose website looks like it’s from the early 90′s) that mislead me thinking the transfer would occur after 7 days. The email read (emphasis mine):

If you wish to cancel the transfer, please contact us before Tue Nov 12 04:13:22 2013 by going to our website, <link removed>

Needless, to say I didn’t click the link until after the deadline passed only then to realize the link had a checkbox I needed to click and submit.

Years ago I’d gotten a Google Apps for Domains account for my Gmail hosted personal email address so once the domain was transferred I just needed to edit the zone file and get it pointed to my WordPress AppEngine project.

Missing Handler

After running into a few problems I found his issue with app.yaml

# 09/29/2013
# https://code.google.com/p/googleappengine/issues/detail?id=9972
# missing handler
- url: /wp-(.+).php
  script: wordpress/wp-\1.php
- url: /(.+)?/?
  script: wordpress/index.php

New Contact Form

After a bit of digging I found a plugin called Contact Form 7 which I’ve used to replace my old BlogEngine.NET contact form.

Conclusion

All in all I’d say this was a pretty damn painful process with a lot of trial & error that took me a few months of late evenings to get going. I’ve been blogging since June 2004 and accumulated an awful lot of cruft over the years that I simply didn’t remember which didn’t help.

The site has now been running for a few days and I’m in the process of trying to reduce the cost of running it on AppEngine as it’s currently over $2/day whereas discountASP.NET I was running about $20/month with SQL support. Just applied a few changes based on this post.

Found this post which I totally agree with.

[Updated: 12/15/2013] Added Contact form info
[Updated: 01/07/14] Updated Google AppEngine plugin to 1.2 running on 3.6.1

WordPress running on Google AppEngine

After several months of slowly migrating all of my blog content and testing Google’s WordPress support on AppEngine I’ve finally pulled the trigger on moving my blog from BlogEngine.NET hosted on discountASP.NET to AppEngine.

w00t!

As time permits I’ll post details here regarding caveats I ran into (some of which I’m still wrangling). At any rate, I’m glad to be off of .NET and have learned a lot about some new parts of Google’s Cloud Infrastructure.

Improving Site Performance

I run BlogEngine.NET and recently logged into Google WebMaster Tools and was a little surprised to read the following paragraph and see the related chart:

Performance overview

On average, pages in your site take 6.8 seconds to load (updated on Apr 4, 2010). This is slower than 82% of sites. These estimates are of low accuracy (fewer than 100 data points). The chart below shows how your site’s average page load time has changed over the last few months. For your reference, it also shows the 20th percentile value across all sites, separating slow and fast load times.

Chart

Yikes! Not. Good.

As you can see I was able to make some quick changes in March that helped but clearly I’m far from being in the sweet spot.

What I Did

Using Google’s Page Speed and Yahoo’s YSlow I started chipping away at the issues. I started on the JavaScript side and added this combine, compress and minify HttpHandler which turned out to be quite easy. Next, I applied Ken Silverman’s pngout utility (warning: his background will hurt your eyes) to my images and minified and combined several CSS files as well as set default images sizes.

Updated April 11, 2010 I’ve turned off Gravatar because it’s too damn slow particularly for pages with lots of comments. That page used to take 10 seconds to load now it’s just a few seconds.

Migrating to IIS7

Over the weekend I took the plunge and used the Migration tool on my discountASP.NET (which btw, I highly recommend) account and moved to IIS7. You can read about how that went here but for the most part things were pretty seamless although my site was down for 16 hours or so. After the migration I setup the MS IIS Manager for remote administration allowing me to set cache expiration headers for lots standard images.

The net result, starting at 78/100…

Page Speed Score 88/100

And from a “C” to an “A” on the YSlow report card:

YSlow performance score

At this point, it’s going to be a few more days before Google’s WebMaster Tools re-executes its performance tests but I’m hoping for some dramatically improved results.

What’s your verdict? How’d I do?

[Update April 13, 2010] This chart from Google WebMaster Tools of download times is starting to show the improvements I’ve made:

image

Switch from dasBlog to BlogEngine.NET

I finally pulled the trigger on switching to BlogEngine.NET. I tweaked it a bit to maintain all of my old blog post links and probably lost a few links here and there but the switch is made. I’m going to run with this for a few days and see how I like it.

Please let me know if you notice anything weird. Btw, if you see any $g(…) macros those were previously Google search links under dasblog which aren’t supported here.

One of the major things I hope this will resolve is the issue of dasBlog losing comments.

[Updated: Nov 24, 2009] I’ve since added the macro functionality found in dasBlog to BlogEngine.NET so the above problem has been resolved.