As I mentioned in my last post, I recently migraded my dy/dx tech website to a different hosting company. If you’ve really been paying attention, you may recall that not too long ago I had gotten a Media Temple hosting account with the plans on migrating all of the sites I host, both my own and clients’ to it only to discover that setting up Django on a Media Temple (dv) account is far more trouble than it’s worth. My estimation of that hasn’t changed, in fact I actually cancelled my Media Temple account a few weeks ago after the last client I had hosted there was moved off to another host. My experiences with WebFaction have been so positive (exploding data centers notwithstanding), that I have instead migrated everything to their servers. Well, not everything yet. This blog is still hosted on DreamHost for the time being (though I plan on moving it to a WebFaction hosted WordPress blog in the very near future before eventually migrating it to a Django based solution as I’ve mentioned before).

The hosting hassles referred to in the title, thankfully, have nothing to do with the actual hosting companies I’m dealing with, and are instead due to a foolish mistake on my part: when I switched my domain to WebFaction, I forgot that I had custom MX records enabling the use of my hosted google apps for my domain. As a result, as the new DNS information started propagating, people stopped being able to send me email. Fortunatly, it was an easy fix to just change the MX records with WebFaction, and I don’t think I missed any important emails, but if anyone out there got a bounceback when sending me an email, that’s why.

It’s been a while since I’ve been able to announce a big new project. Not because I haven’t had any, but because everything I’ve been working on lately has been so large that nothing is quite ready to go live yet. But finally, I get to announce a big project that I recently finished: the Becoming MOBOS video blog. As I’m sure many of you from the Boston area are aware, there is a new Mandarin Oriental that’s been under construction down by the Pru. They hired me to create an internal video blog for them. Unfortunately, since it’s internal, I can’t link to it, but the screenshot to the right links to a full-size, albeit redacted, image. It’s a WordPress based blog using a verstion of WPelements.com’s MassiveNews theme customized by your truly. I also used FlowPlayer to provide the Flash video playback capabilities. All in all, I think it turned out to be a pretty slick site.

That’s not the only news, however. In preparation for announcing the Becoming MOBOS site I’ve been doing a little work sprucing up my own website. So I also get to announce a new version of the dy/dx tech website (I also changed hosts for it, so you may need to wait for the DNS to propagate if you’re still seeing the old site). The overall look of the site is the same as before, but I’ve removed some rather pointless elements such as the Google Map that used to be on the front page. In it’s place is now a slideshow of screenshots from my portfolio, which I think is a much better use of the space. The majority of the changes, however, are under the hood. As you may recall, I redeveloped the site using Django a while ago. Since then I’ve spent a lot more time with Django and know a lot more about it, so I completely redeveloped the site (using the newforms-admin branch and was able to make a lot of improvements to the code, and basically leave it better positioned to integrate more features in the future. Among other things, I plan on migrating this blog to a Django-based solution and integrating it into the dy/dx tech website to some extent. I’ve been working heavily with Django for the past several months, and I just keep liking it more and more. It makes every part of my job so much more enjoyable and, in a lot of cases, faster. Be on the lookout for another project going live in the next couple weeks: this one will be Django-based and will be very public, and, I predict, very popular.

Currently I’ve got two projects hosted on WebFaction servers. So far, I really like them. As managed hosts go, they’re probably the best I’ve worked with, and they certainly make life very easy when building Django powered sites.

Today I got an email from one of the clients whose project is hosted on WebFaction saying that their site is down. So I checked it out, and while I was able to access it, it was extremely slow, to the point where a less forgiving browser/LAN setup might cause it to time out. So I fired off a support ticket to WebFaction, and within a couple minutes, not only was the site back up to speed, but I was provided with a very good explanation for why my server was having problems.

Apparently there was an explosion at one of WebFaction’s data centers this weekend. It took out power to the data center, but fortunately no one was hurt and none of the servers were damaged. Obviously, there have been some interruptions in service for the servers in that data center (which includes both of my WebFaction projects), but they’ve already gotten a significant number of the servers back online (though only one of mine).

Amazingly, this is actually the second time I’ve had a server taken out by an explosion at a data center. The first time was with a hosted Microsoft Exchange server with a hosting company in London.

It really sucks having sites down, especially critical ones (fortunately only one of the projects I have hosted with them is critical, and it’s the one that’s back up already), but as reasons for downtime go, you have to admit that an explosion is a pretty good one.

I’m currently working on a fairly large Django project that I think I’ve mentioned a couple times in the past. In that Django project there is a Person model and an Organization model. Both Persons and Organizations have email addresses. Organizations are related to Sites (through a ManyToMany field), but Persons are not (they’re related to Organizations through an intermediary table). I’m currently attempting to create a contact form, so that people can email either a Person or an Organization using newforms.

For security reasons, we don’t want the user to actually see the email address, just the name of the Organization or Person. Fortunately, newforms has the ModelChoiceField class that you pass a queryset of options. In keeping with DRY principles, I want to be able to use a single Form regardless of whether the person is trying to email a Person or an Organization (both models have a field named ‘email_address’). Unfortunately, this is where I ran into my first problem: ModelChoiceField doesn’t really allow you to define the queryset dynamically, you have to define it in the form definition. Luckily I found this blog entry which provides a method to re-define the queryset in the __init__() method which allows you to change it based on the HttpRequest object. My next problem was that I only want the user to be able to email Organizations and Persons on the current Site. Since Organizations are directly related to Sites I just used the CurrentSiteManager. However since People are not directly related to Sites, and are instead related only to Organizations (through an intermediary table), I couldn’t do this. Instead, I decided to try this crazy bit of code:

Person.objects.filter(persontoorganization_map__organization__in=Organization.on_site.all())

Amazingly, it just worked exactly as I would have wanted it to. No fuss, no problems, just a queryset of Persons related to the current site. Go Django!

As you may recall, a while ago I got myself an account at MediaTemple with the idea that I’d move all my websites over to there. I had previously been using Dreamhost, but wanted something a little more high quality so that I could reasonably offer hosting services to some of my clients. MediaTemple seemed like a good way to go, and for the most part their service has been great.

Unfortunately, I have run into a few problems. Most importantly, despite spending a fairly significant number of man-hours working on it, I’ve been unable to get Django running on my (dv) server. Yes, they have a (beta) program that makes it easy to run Django on a (gs) account, but for my needs a (gs) simply won’t do and I really don’t want to have multiple accounts with them. The end result of this is that several of my web pages are still running on Dreamhost because they require Django (this blog actually is as well even though it’s currently a WordPress blog, because I want to switch to something Django-based and it seems like an unreasonable hassle to migrate my WordPress blog to a new server only to then have to migrate it again to new software, especially as I’m currently holding onto my Dreamhost account for my Django-based pages anyway).

The issue is now coming to a bit of a breaking point. Why? Because I’m currently working on a pretty large Django-based website that will be going live in the next month or two. For the purposes of development, it’s being hosted on WebFaction, which has been an amazing host. They make it incredibly simple to host a Django site, to the point that basically zero setup is required. But as we get closer to the point of going live, I’ve been considering what the hosting needs of the site will be going further, and how to best serve them.

The site is a redevelopment of an existing site, so we can get a pretty good idea of what the traffic numbers are going to look like. This will let us extrapolate the RAM and bandwidth requirements pretty well too. The issue, is that once the Django version of the site goes live, we’re going to start to expand it. Thanks to the capabilities of Django, it’s being developed with the potential for massive growth in mind. Specifically, it’s using Django’s Sites framework to allow for expansion to several sites. Currently there are only two, but the Django version will go live with 4 or 5, and there’s the potential to expand far beyond that.

This means that we’re going to require a pretty large number of (software) servers. There’s the MySQL server running the back-end, an HTTP server for static content, and then an HTTP server for each site, and they’re all going to be using up resources to different extents. Trying to find the best hosting solution for this sort of setup has led me to a couple of options:

1. Stay with WebFaction. A number of people have said they believe that WebFaction’s shared hosting plans should be able to accomodate this. WebFaction provides a very good combination of ease of use and low-level access, their prices are good, and the way they have their hosting set up, it’s extremely simple to add another Django install complete with its own Apache/mod_python instance. They also offer dedicated server, but I think there are probably better routes to go than with WebFaction’s dedicated servers.
2. Switch to Slicehost. I’ve only just learned about Slicehost, but so far they look like a pretty sweet deal. For a very reasonable price you get a virtualized server running a Linux distro of your choice (you can choose from 8 right now) run on Xen. They claim not to oversell their servers, so you’re guaranteed to actually get the full capacity that you pay for (unlike with budget hosts such as Dreamhost). And since you’re getting your own virtualized host you have full root access. They basically have nothing preinstalled, so you can easily set it up in whatever configuration you want without having to deal with the vagaries of the anointed hosting package (Plesk on MediaTemple, I’m looking at you). I really like like look of them, and they’ve been getting good reviews. They’re currently listed as the number 2 hosting company on Djangofriendly, behind only WebFaction. With my background in IT and Linux administration, the fact that I’d have to manage everything myself isn’t enough to scare me away either. The fact that they let you choose your Linux distro really appeals to me as well, as I’m by far more familiar and comfortable with Gentoo than any other Linux distro. I only wish they offered FreeBSD slices, but the only reason they don’t is technical, and once that issue is resolved it sounds like they plan on it. In a lot of ways, they’re basically a very affordable colocation provider. The biggest issue, it sounds like, is that apparently communication between different slices counts against your bandwidth allotment (for both slices, presumably). This means that as the site I’m working on grows, if it spreads out to multiple slices (which it undoubtedly would and which I’d want it to do since that will give the added reliability of spreading across multiple physical machines) we’ll basically be billed for database access from the sites that aren’t on the slice with the database server.
3. Colocation. Colocation is basically the 800 lbs. gorilla in the room. It costs a lot more, but you get what you pay for. With colocation we’d have all the advantages of Slicehost (minus the low price, of course) plus the ability to expand more or less arbitrarily. We could have as many physical machines as we wanted running as much or as little of the site as we wanted. Provided we’re willing to pay, of course. On top of that all the server management would again fall to me, but this time without some of the nice shortcuts that Slicehost offers. Essentially, colocation is alway the fall-back option. But hopefully one of the other two hosts can offer us a solution that’s a little more balanced: we get less control, but more simplicity and ease of use for a greatly reduced price.

At the moment, I’m leaning towards sticking with WebFaction for now. I’m already very impressed with what they offer, and from the sound of things, they’ll continue to be a more than adequate host as we expand. But I’m also definitely looking for input. If anyone has any suggestions or recommendations I’d love to hear them. In particular, any first-hand experience with hosting large Django sites with any of these solutions are most welcome.

So, a little while ago I got a MediaTemple (dv) server. In general, they seem to be pretty good. The server’s been fast and reliable so far, I get a good amount of storage and bandwidth for the price, and I have root access so theoretically I can do pretty much whatever I want. I have run into a bit of a problem however.

For the past several days I’ve been banging my head against installing Django on my MediaTemple server. In theory this should be pretty easy. The server comes with Apache and mod_python installed, so all that I should need to do is check out the lated django trunk from svn, make a few symlinks, install MySQLdb, change some Apache configurations to tell it to use Django for the appropriate URLs, and go. In practice, it’s not working quite so well.

The first few steps were simplicity itself. It wasn’t long before I had a successful Django install and was able to ‘python manage.py syncdb’ to have my projects database tables created. Changing the Apache settings was a little more difficult because you can’t actually modify the httpd.conf as Plesk will overwrite it. Of course, there’s no documentation telling you this or telling you the appropriate solution (despite, I might add, the insistence of the MediaTemple KnowledgeBase that instructions for installing Django on a (dv) server exist somewhere). A little help from Django’s Google Groups community helped me out with that, though, letting me know that I instead need to create a /var/www/vhosts/MYDOMAIN.COM/conf/vhosts.conf file and put the settings in there. So I did that, and it even seems to work, but when I try to actually go to the page I get this:

Mod_python error: "PythonHandler django.core.handlers.modpython"

Traceback (most recent call last):

  File "/usr/lib/python2.3/site-packages/mod_python/apache.py", line 299, in HandlerDispatch
    result = object(req)

  File "/usr/lib/python2.3/site-packages/django/core/handlers/modpython.py", line 188, in handler
    return ModPythonHandler()(req)

  File "/usr/lib/python2.3/site-packages/django/core/handlers/modpython.py", line 161, in __call__
    response = self.get_response(request)

  File "/usr/lib/python2.3/site-packages/django/core/handlers/base.py", line 64, in get_response
    response = middleware_method(request)

  File "/usr/lib/python2.3/site-packages/django/contrib/sessions/middleware.py", line 15, in process_request
    request.session = engine.SessionStore(session_key)

AttributeError: 'module' object has no attribute 'SessionStore'

Apparently, for some reason, there’s some problem with Django’s sessions middleware, although no one else seems to have discovered this problem. If I remove sessions (and therefore the admin app) from my project the page will actually load, but it then fails to actually be able to get anything from the database so gives me errors on any page that requires database calls (most of them). Thus far neither the Google Groups nor the expert aid of Jesse Legg have been able to help me make any progress in solving this issue.

There are people out there running Django on MediaTemple (dv) servers, so I know this is possible, and I’m confident that I’ll get it eventually. But in the mean time it’s incredibly frustrating and is putting serious delays in the process of transitioning my sites over to MediaTemple (not to mention in the development of some other Django projects I’m working on right now). You can be sure that when I finally get Django working on my (dv) server that I’ll be writing a detailed account of exactly how it was done. There really needs to be some documentation on this out there.

I’ve been meaning to do it for a while, but I’ve finally gotten around to creating a portfolio. Since my website is now Django based this was incredibly easy, and probably only took about 45 minutes to do. Previously I’ve just been keeping a list of links to former projects that I included in emails to prospective clients. I didn’t really want to put up a portfolio when I only had a handful of projects to show off. But in the past few days three different projects went live: Lola Boston for which I created the locations database, Tundratour for which I created a database for the different trips as well as a ’shopping cart’ to allow people to request more information on multiple trips at once, and Sel De La Terre which was live before but now has a tool for purchasing gift cards online that I created. I figured that 5 projects was a big enough number to go live with, especially as it should be growing pretty rapidly in the next few months.

I’m still trying to decide if I want to put more information in there. Should I put in a little description of each project, or at least expand on what my contribution to the project was? The Chainsaw Awards page was nominated for the MITX awards, I should probably mention that somewhere. Fortunately, now that it’s there it will be easy enough to add more to it. The other question I’ve been asking myself is whether or not I should include websites that I worked on at my old job. I was just as much responsible for those projects as the ones I’m doing now, but somehow it just seems like I should leave them off. Fortunately again, it will be easy to add those later if I decide to. And in the meantime I have a portfolio to show off. A pretty nice one, if I do say so myself.

Also, I do still intend to write that post that I promised while I was in Jamaica, I’ve just been very busy ever since getting back. I’m going to be away again this weekend (Florida for another wedding), so hopefully I’ll be able to get to it next week when I’m back.

When I first decided to start my own business, my idea was basically to offer IT services for Mac using individuals and companies. Previously I had been working in IT at an all Mac corporation, so I definitley have the skill set to do this. The problem with that idea turned out to basically be one of supply and demand: there’s quite a lot of competition in the field, and as a newcomer without much of a background it was hard to distinguish myself and actually land jobs. During the past 18 months or so that I’ve been doing this I’ve gotten some work doing IT, but not really all that much. Instead I found myself filling the gaps by doing web development. For whatever reason, it appears that I’m much better at selling myself as a web developer than I am at selling myself as an IT consultant. So I’ve found myself doing mostly web development with IT work pretty much being something I do on the side.

So in the past month I’ve made the decision to change my business focus. Instead of dy/dx tech being a Mac IT business that also does some web development work, it is now a web development business. I’ve even let my membership in the Apple Consultants Network lapse, as the main benefit I derived from it was the great prices for software which I’ve now already got. There are a number of benefits for me making this switch. The first is that I’ll now be able to focus all my energy on finding web development work rather than it being a secondary focus. Since even as a secondary focus, web development was much more successful than IT, I think this will really pay off. The other benefit is that it will be easier to distinguish myself from the competition. I have a number of web site projects that I’m working on right now, and that I’ve worked on in the past that I can put my name on. Having that lets me put together a nice pretty portfolio of work, something that IT work just isn’t as good for. I’m currently working on putting together a portfolio application in Django to add to my website. This will increase the ability of my website to sell my services, and also serve as visual evidence of my businesses growth and therefore as a rought metric for the quality of my services. I’m waiting to finish up a few of the bigger projects that I’m currently working on before going live with online portfolio.

I will still, however, be doing some Mac IT work. But now I’ll be doing it under the In Home Mac brand. In Home Mac is a company started by Matt Moglia, a good friend of mine from High School. He started doing the Mac IT thing for himself in the Bay Area at about the same time I started doing it in Boston. He’s apparently much better than I am at marketing those services, and has now built up his company to the point where he’s got multiple techs working for him in different areas. I’m now the In Home Mac tech for the Boston area. The main advantage of this for me is that I no longer have to worry about advertising those services. All the advertising and such wil be taken care of for me, so I just have to take care of the work when it comes in. This is just about a perfect arrangement for me as I can focus on selling the services that I’m actually good at selling and still have work to do with the services I’m good at but can’t sell well.

I’m pretty confident that this change will, in the next several months, lead to me bringing in a lot more busines than I have been. Especially as I’ve already got a pretty good business model for the web development side of things. It’s pretty rare to find people who are both good designers and good programers. This makes a lot of sense when you think about it, but what most people don’t realize is that both skill sets are necessary for functional websites. In general, I’ve found that web designers tend to do the designing and then either muddle through with what little programming skills they have or get a friend or relative who knows more programming than them to do it on the side. This works for a while but tends not to be a very scalable model for a number of reasons. First, someone who’s doing web development on the side can often make time to get one project done, but they just can’t do it for project after project because they have other things to worry about. Second, it’s usually not reasonable for that person to quit their day job and do web development full time because a single web designer or small web design firm isn’t likely to generate enough work to justify a full time developer. So I have an advantage here. I have as much time as I need to dedicate to web development. On top of that, I have enough time to fine more web designers to partner with. The more designers I work with, the more steady the work coming in will be. Currently I’m working with two different small design firms which brings in decently steady work, and I’m always on the lookout for more designers and small design firms that might be interested in hiring me to do their development. This basically ends up being win-win for everyone as several different designers get to have a developer without needing to hire one full time, and I get to work full time as a developer without having to sell my soul to a big company.

My eventual plan now is to partner with enough designers and design firms that I can’t actually handle all the work myself. At that point I can bring in more developers in the same way that Matt has brought in more Mac techs and my business can really start to grow not only in size and revenue, but in the variety and quality of the services that I can offer. There’s only so much I can do myself, but when I start bringing in more people I’ll be able to add their unique skills to the services I can offer and I’ll be able to delegate work out in a logical way to improve the workflow, efficiency, and quality. I’m not quite to the point of needing to do that yet, but I think I’m on track to get there. And I’ve already got a few other people in mind to bring into the dy/dx tech web design fold including an SEO specialist and another general developer who’s good with Ruby on Rails.

This will be my second post written from my iPhone. I’m currently in the middle of nowhere in Illinois at Jessi’s mom’s house. We’ll be here until the 30th, which means I’ll have some work to do on the trip. On my previous visits here there was always a wifi network available from one of the neighbors who was kindly sharing his sattelite Internet connection with the rest of the town (it’s a very small town and they can’t get cable or DSL out here). Sadly it turns out that someone had been using up his monthly bandwidth allotmen so that’s no longer available to me. Hence me writing this entry from my iPhone.

Fortunately, I came prepared to work without Internet access. I had planned to do some work on the flight over here, so before I left I created a new Parallels virtual machine and installed a copy of Gentoo Linux in it, and set up a lighttpd server with PHP and Python running in FastCGI as well as SQLite and MySQL so I could do both PHP and Django development without having to only guess at whether or not it would actually work (technically this wasn’t necessary for Django since it comes with a lightweight development server built-in, and OS X comes with SQLite installed standard, but I figures I might as well).

I also tried setting up Tinyproxy on my iPhone so I could share my iPhone’s EDGE connection with my laptop for browsing, but for some reason it doesn’t seem to be working even though I had no problem with it at home.

The end result is that, even though I have no real Internet access to speak of, I can still do my work. I can even keep in touch with everyone thanks to my iPhone and meebo, and, if the need were to arise, I could always set up an SSH tunnel through my iPhone and get SFTP access to my webserver (or any other, for that matter) that way. So even way out here where they don’t even have DSL, I’m still fully connected. This is one of those times where, even though I spend pretty much all my time working with technology, I’m still completely amazed by it.

Now, I believe, it’s time for dinner.

My first Django project is now live. Sadly it’s not a very interesting one, just a re-develop of the dy/dx tech website. It looks exactly the same as before, but it’s now powered by Django. This doesn’t really offer any advantages at the moment, but it will. For example, pretty much all the data on the site is currently stored in a database which means it will be extremely simple to add, remove, or change any of the services show in the services tab. Not that I really expect that to change any time soon (although you may notice that the services tab is the one part of the website that has changed; I’ve added a few, and consolidated some redundant ones), but the principle is sound.

More importantly having the site powered by Django will make it much easier for me to add some new features/online services that I’ve been thinking about for a while. The first one will definitely be a portfolio tab. I’ve worked on a pretty good number of websites in the year or so since I started this business, and I really should have a portfolio on my website to show off my work. I’d also like to put up a clients tab where I can list my clients and, if I’m lucky, get some testimonials to put up there as well. Then of course there’s the WiFi database that I’ve been talking about for a while, that will have it’s own subdomain, but I’ll give it its own tab as well. Hopefully now that the whole site is done with Django and I’m a little more familiar with how the framework works development of those things and others will go a little faster.