I've been nothing but impressed with the service I've got from WebFaction and the reliability I've gotten from their servers. I even had a very high traffic site run on a WebFaction shared account without a hitch.

Today, however, we got a first hand look at the downside of a shared server. If you're a webfaction customer you should be (no really, you should) subscribed to their status blog It's a great way to be kept up to date on any issues that might affect your site. The most recent issue has to do with the MySQL server on web49: the server that we happen to be using to develop a very large project. As you can see from reading the entry, the problem appears to have been caused by a corrupted database table (not one of ours) which was causing some unreliability with the database server (our Django based site was intermittently unable to connect to the database and, when it could, intermittently unable to authenticate). Though they thought they had it fixed, the problem returned and while they're attempting to fix it for good they've rolled back the entire database server to a known good backup.

This is a good approach as it means that everyone should still have most of their data in the meantime. Unfortunately, we happen to be in the middle of populating the database with the data we need to go live in the near future. Rolling back the database even by a day means that we've lost a ton of work. We should get it all back once the problem is fixed, but of coruse that means that we have to put the work on hiatus until the problem is fixed to avoid versioning issues.

This right here is the perfect illustration of why a dedicated server is a good idea. Yes, a shared server might be able to support your site. But it also leaves you vulnerable to the actions of the people you're sharing the server with. If someone else does something stupid that corrupts a database table on a server that they share with you suddenly you stand to lose a lot of work. If someone has a poorly written app that somehow manages to crash the server or even just eat up all the RAM, your site goes down. With a shared server you simply don't have the security of knowing that your site is stable and secure even if you trust your hosting company and you trust your code.

That security is what you're paying for when you get a dedicated server over a shared one.

I've been using WebFaction for my hosting for a while now, and have been extremely pleased with them. In addition to the fantastic service I've received, I've been very impressed with the intelligent way they have their servers setup. They've clearly done a lot of work to make things as modular as possible which makes it insanely easy for me to run multiple sites with very different requirements seamlessly on the same server.

Basically what they've done is segment out all of your different websites into 'applications'. Each application in represented as a directory in your ~/webapps/ directory, and is essentially a self-contained environment with it's own apache instance, and, in the case of a Django app, it's own $PYTHONPATH. The end result is that even though all the websites are being stored and run from within my home directory, they're entirely modular, can have different, or different versions of the same, dependencies installed, and can be shut down and restarted independently of one another. On top of all this is a fantastically simple custom web-based control panel that I'm pretty sure is built with Django.

I've been so impressed with how well this setup works, that I've decided to duplicate it on my home server for development purposes. Currently I do pretty much all my development work on my Gentoo Linux powered ThinkPad. To that end I've installed Apache, MySQL, PostgreSQL, SQLite, Python, PHP, &c.; to allow me to mimic the live sites as closely as possible and to allow me to continue working when I don't have internet access (such as when I'm flying or visiting Jessi's family out beyond the reach of broadband). This works very well, but as I'm just using a basic Apache install, without any VirtualHosts, it's not nearly as flexible and means I can really only work on a single site at a time with some work necessary to switch back and forth between projects. Of course most of the time I just use Django's built-in development server when working with Django, but I do end up relying on Apache sometimes, and I'd like to set up my home server as a more complete development environment for both myself and some friends I can grant VPN access to. So to that end I've been looking into WebFaction's setup with the idea of re-implementing it myself.

Turns out it's pretty simple. Simple enough that I almost feel like I should have thought of it myself. Basically, WebFaction's setup scripts create a new 'app' in your ~/webapps/ directory, and populate it, most importantly with a copy, owned by your user, of the Apache executable, some scripts to start, stop, and restart that executable, and an httpd.conf file that sets the (in the case of a Python-based app) $PYTHONPATH variable to include a ~/webapps/yourappname/lib/python2.5/ directory allowing each site to maintain it's own dependencies independently (you can also put things in your ~/lib/python2.5/ for global dependencies if you want). Oh, each application also gets it's own copies of the necessary Apache modules to the same effect. Each application's Apache instance(s) is set up to listen to a different (non-80) port. The end result of this is an extremely simple, extremely modular setup that works fantastically.

Obviously I've left out a step here. If each Apache isntance is listening to a different, non-80, port, how does your traffic get to your actual site? This is the one part that I can't really just peek into the configuration files for, because it doesn't (as far as I can tell, which makes sense) live on the same server as my sites. I assume that what's happening is that WebFaction's name servers are simply pointing requests to (for example) joshourisman.com:80 at my.webfaction.server:portnumber. Again, a simple, yet elegant solution that allows for easy customization and expansion.

I haven't yet tried to implement this setup myself (I first want to move my server from FreeBSD to Linux (which now that I'm using full-time again I'm just much more familiar with), but there's nothing about it that's particularly tricky. Really, the routing is probably going to be the hardest part, but I'm planning on replacing our rather lackluster TrendNet wireless router with a Linux box which will give me much greater control and (hopefully) better reliability.

Yes, I've managed to import the old posts from my blog! It was pretty easy to do, complicated only by the fact that I initially accidentally pulled the data from a different, older WordPress blog that I deleted some years ago but apparently still had the MySQL databases for.

At any rate, all my posts are now once again accessible, and any old links to them should still work. Comments have not yet been imported, but that's the next step. For now everything should be working as expected, but please let me know if you encounter any errors or problems.