Storing IP addresses as integers in Python
Oct 7th, 2009

I just saw this very interesting (for a programmer) blog entry on Storing IP addresses as integers. As the article says, anyone who wants to store IP addresses in a database is generally going to do as a string. However if you're really concerned about memory efficiency, you might want to find a lighter data type to use. Since an IP address is really just a collection of integers it would make sense to store it as an integer, however doing so without losing important information (specifically, the placements of the dots) becomes an issue.

The blog post in question introduces the the pack() and unpack() functions, which 'allow you to create and extract data into and out of binary-packed strings' and provides the Ruby code necessary to encode an IP address as a simple integer and decode it back to a dotted quad. I thought this was pretty cool, so I decided to write the equivalent Python code. This is what I ended up with:

It seems to work as advertised, though the packed string the encode() function returns is different from what Ruby gives you, so they won't interoperate (it would probably only require changing the pack() and unpack() formats to do so, but I didn't feel like experimenting to figure out which ones).

Of course, being a Django guy, the immediate application of these functions that I see is creating a new IPAddressField that stores the address as an integer instead of a string transparently.

Hours of Operation
Dec 16th, 2008

As you probably know, I've been working on a Django-based re-build of BostonChefs.com (the new version of which is actually live now, but due to DNS propagation issues isn't yet available to 100% of people which is why I haven't yet written a post about it). Among other things, BostonChefs.com provides information on some of the fantastic restaurants in the Boston area. One piece of information it provides is the hours of operation of those restaurants. In order to store this information I created a model called HoursOfOperation. It looks like this:

class HoursOfOperation(models.Model):
    DAY_CHOICES = (
        ('0', 'Sun'),
        ('1', 'Mon'),
        ('2', 'Tue'),
        ('3', 'Wed'),
        ('4', 'Thur'),
        ('5', 'Fri'),
        ('6', 'Sat'),
        )
    restaurant = models.ForeignKey("Restaurant")
    meal_period = models.ForeignKey("MealPeriod")
    day = models.CharField(max_length=3, choices=DAY_CHOICES)
    open_time = models.TimeField(default=datetime.datetime.now)
    close_time = models.TimeField(default=datetime.datetime.now)


    def _get_hours(self):
        return "%s - %s" % (self.open_time.strftime('%I:%M%p'), self.close_time.strftime('%I:%M %p'))
    hours = property(_get_hours)

As you can see, each 'hour' is related to a restaurant and a meal period, which allows us to display the information in a manner similar to that you might find on a store's front sign. For example, if you go to the Grill 23 & Bar page (my personal favorite restaurant in Boston, although Craigie on Main is a decent challenger), you'll see something like this:

DINNER
* Sun: 5:30 p.m.-10 p.m.
* Mon-Thur: 5:30 p.m.-10:30 p.m.
* Fri: 5:30 p.m.-11 p.m.
* Sat: 5 p.m.-11 p.m.

Building a list like that out of the above model proved slightly more difficult that I might have hoped. It required quite a lot of template logic, including writing a custom filter. The block of template code necessary to generate that list looks like this:

<div class="hours">
    {% regroup restaurant.hoursofoperation_set.all by meal_period as periods %}
    {% for period in periods %}
    <div class="hoursMealPeriod">{{ period.grouper }}</div>
    {% regroup period.list by hours as hour_list %}
    <ul>
        {% for hour in hour_list %}
        <li>{{ hour.list|collapsedays }}</li>
        {% endfor %}
    </ul>
    {% endfor %}
</div>

As you can see, somewhat complex. Those nested {% regroup %}s can be nasty to wrap your head around, if nothing else. But basically it's taking the set of HoursOfOperation objects related to the restaurant, grouping them by meal period, then taking the subset of those objects for each meal period, and grouping those by the hours of the day they represent. So what you're then left with is a list of all the different time periods (still represented as HoursOfOperation objects) that the restaurant is open for a given meal period, and the days on which it is open during those hours. As you can see above, the days are represented by number of the day of the week (0 for Sunday through 6 for Saturday).

Converting that list integers into something like 'Mon, Wed-Fri' was not very easy, and certainly not something I wanted to try to tackle using Django's template tags. I ended up drawing heavily on my hazy memories of CS 127 (many thanks to Dave who taught me all about recursion way back then) and creating a filter that considers the list of HoursOfOperation objects as a list of those integers, then recursively converts it into a list of lists representing the subsets of contiguous days in the list. So if you start out with [1, 3, 4, 5] you end up with [[1, 1], [3, 5]] which is then converted into 'Mon, Wed-Fri'. After several false starts I ended up with this beauty of a Django template filter:

from django.template import Library
from django.template.defaultfilters import time


from types import ListType

register = Library()

def simplify(index, found, days):
    high = index+1
    mid = index
    low = index-1
    if not found:
        days[low] = [days[low], days[low]]
    if high >= len(days):
        if not isinstance(days[-1], ListType):
            if days[-1] == days[-2][1]:
                days.pop(-1)
            else:
                days[-1] = [days[-1], days[-1]]
        return days
    if int(days[high].day) - int(days[mid].day) == 1 and (found or int(days[mid].day) - int(days[low][0].day) == 1):
        days[low][1] = days[high]
        days.pop(mid)
        high = high-1
        found = True
    else:
        if found:
            days.pop(mid)
        found = False
    return simplify(high, found, days)

@register.filter
def collapsedays(value):
    hours = "%s-%s" % (time(value[0].open_time), time(value[0].close_time))
    days = simplify(1, False, value)
    for i in range(len(days)):
        if days[i][0] == days[i][1]:
            days[i] = days[i][0].get_day_display()
        else:
            days[i] = "%s-%s" % (days[i][0].get_day_display(),
                                 days[i][1].get_day_display())
    return "%s: %s" % (', '.join(days), hours)

Calendar-based blog archive
Sep 21st, 2008

Over to the right, in the sidebar, you may notice a nifty javascript calendar displaying the current month with some days highlighted. It is, in fact, a navigation tool for the blog. The highlighted days are the days with posts. Click on a day, see the posts for that day. If there are no posts for that day, it falls back to the month. If there are no posts for that month, it falls back to the year. Pretty simple stuff, really.

The calendar is implemented with YUI, and the fallback mechanism is, of course, all Django code. The fallback isn't something that's built into Django, sadly, but was relatively easy to implement by just wrapping Django's date-based generic views in some views of my own. Really it's just a simple 'try except' block that catched a 404 error and instead returns a redirect. Not too shabby, if I do say so myself:

from django.template import loader
from django.http import Http404, HttpResponseRedirect
from django.views.generic import date_based


def archive_month(request, year, month, queryset, date_field, month_format='%m', template_name=None, template_loader=loader, extra_context=None, allow_empty=False, context_processor=None, template_object_name='object', mimetype=None, allow_future=False):
    try:
        return date_based.archive_month(request, year, month, queryset, date_field, month_format, template_name, template_loader, extra_context, allow_empty, context_processor, template_object_name, mimetype, allow_future)
    except Http404:
        return HttpResponseRedirect("/%s/" % year)

def archive_day(request, year, month, day, queryset, date_field, month_format='%m', day_format='%d', template_name=None, template_loader=loader,extra_context=None, allow_empty=False, context_processor=None, template_object_name='object', mimetype=None, allow_future=False):
    try:
        return date_based.archive_day(request, year, month, day, queryset, date_field, month_format, day_format, template_name, template_loader, extra_context, allow_empty, context_processor, template_object_name, mimetype, allow_future)
    except Http404:
        return HttpResponseRedirect("/%s/%s/" % (year, month))

(Also check out nifty syntax highlighting! Based off this snippet from Django snippets, modified to remove the Markdown bits. And by highlighting, I of course mean formatting. I'll be adding fun colors and such to it in the near future.)

My first Django patch
Sep 16th, 2008

I just submitted my first patch to Django! Among other things, this is my first real forray into the inner depths of the Django code. This patch fixes an issues that had been bothering me for quite some time. In Django's admin interface it's possible to specify that a particular field should be automatically filled in with the value(s) you enter in some other field(s). For example, as I typed 'My first Django patch' into the title field of the form I used to write this post, it was automatically filling in a slug field that's being used for the permalink to this post with 'my-first-django-patch'. This is a very useful features and uses just a little bit of javascript to accomplish it. The only problem is that it only works when you're trying to pull information from a text field. Sometimes, however, you might want to pull information from another sort of field, such as a drop-down menu. Previously Django simply wasn't capable of this. With my changes, however, it is able to handle this potentiality quite well.

It's not really a huge patch, just a fairly a little added code to a single javascript method, and there's no guarantee that my patch will every make it's way into the Django code base, but it's still fun to be able to contribute to one of my favorite open source projects ever. The patch, for those that are curious, can be found here, and I've also submitted it to djangosnippets.org here.