Archive for August, 2007

August 31, 2007

Getting the Time Zone from a Web Browser

Writing rich date/time features in a web app can be a pain. Apps (such as schedulers) that do math on times (e.g. ordering times) should pay attention to time zones for those times, but it’s difficult to know which time zone should be used to display the times to the user. Asking the user to explicitly choose a time zone is natural and often necessary, but a long list of time zone choices can be intimidating to the user. I’ll discuss one method you can use to detect the probable time zone of a browser. It’s not perfect, but it offers a good default (and it’s easy to code.)

Time zones

For apps that don’t have rich date/time functionality, times can be represented as simple numbers or strings. For example, if I wanted to meet you in San Francisco to go to the late showing of The Bourne Ultimatum at the Metreon theater, it’s probaby fine to say “let’s meet at the theater at 11:15 PM on Friday; the show starts at 11:30 PM.” Since we’re in the same time zone as each other, and as the theater, we don’t really care about time zones. An application that’s facilitating this interaction could store and display the times “11:15 PM” and “11:30 PM” without regard to time zones.

Metreon

If we want to do date/time processing on the back end (e.g. ordering events in time), it’s more general to store absolute times in the database. For example, instead of storing something like “hour => 11, minute => 15″, which might mean different absolute times to different users, it is convenient to store a canonical time such as the number of seconds since January 1, 1970 in GMT. That way, we can compare all of the times in our database without having to worry about the time zone for each.

GMT

If we store “absolute” times in the database, then displaying times to users becomes a localization issue. It’s pretty similar to localizing web site content into the language of the user. You start with a notion of what you want to present to the user, then you identify how the user wants to receive it (i.e. their language/locale), and then you localize the content at runtime (i.e. you show them the content in their language.)

As with language, you need to know about the user. The HTTP protocol specifies that the “Accept-Language” header can be used by servers to find out which language(s) the user prefers. The “Accept-Language” header is nice because it lets websites show content in the “correct” language without having to explicitly ask the user. A user who only speaks French doesn’t have to puzzle through an English language page that says “click here for French” somewhere in a footer- they just see the content in French. Even better, it’s one less setting that the server has to manage, and that the user has to set and keep up-to-date.

Unfortunately, there is no corresponding “Accept-Timezone” header- the HTTP standard does not contain any facility to allow the browser to automatically tell the server what time zone the user cares about.

There are two standard ways for developers to deal with this.

First, they can ignore it. For many apps, this is a decent approach- just store “11:30 PM”, and don’t worry about the time zone. As long as all the users who care about that time know what time zone it’s in, then the app doesn’t have to keep track of it.

Second, they can ask the user to make an explicit choice. For example, when setting up Google Calendar, you are asked to choose a time zone. That’s fine for the developer, but finding the “right” time zone in a long list can be a pain for the user.

I wanted to let users choose a time zone on my site, but I also wanted to have an intelligent default- for most users, they shouldn’t have to take any action; the choice I make for them should be correct.

This calls for Javascript on the client. I wanted to write some Javascript that would choose the right option in a time zone dropdown.

This is slightly harder than it seems because Javascript ALSO does not contain a way to get the time zone of the user. Javascript DOES, however, provide a way to get the offset from GMT for any particular time. A time zone can be thought of as a rule that says what the GMT offset is for different times. We can therefore do a reverse mapping- if we know the GMT offset for a few times, we can figure out the time zone for the user. Time zones can be quite complicated (some include Daylight Savings Time, some start or end DST on different dates than others, sometimes the DST offset isn’t a full hour, etc. There are even time zones that are identical for all FUTURE times, but had differences in the PAST.)

In theory, we could deal with all of these cases by doing many probes- we could check the GMT offset for many times, and get an exact time zone match. In practice, this really isn’t necessary- most users are in the more populous time zones, and the cost of failure (defaulting to a time zone that’s similar but not quite right) is not terribly high. Instead, we can probe two times (one in the summer and one in the winter) to find out the normal GMT offset, whether the time zone has Daylight Savings Time, and the DST offset.

In terms of implementation, I wanted to basically make a list of recognized offsets. That is, a list that says “if the summer offset is -7 hours, and the winter offset is -8 hours, then the time zone is probably US/Pacific.”

I like hacking in Ruby, so I grabbed the TZInfo Ruby library, and wrote some code to run through the known time zones, figuring out the winter and summer offsets for each. After grouping by offsets, I had to choose a winner in the case of duplicates. When multiple time zones had the same summer and winter offsets, I searched for each of them on Google. I figured that the time zone with the most hits was probably the most popular one, so I chose that one. Here’s the Javascript code that I came up with:

function getTimezoneName() {
	tmSummer = new Date(Date.UTC(2005, 6, 30, 0, 0, 0, 0));
	so = -1 * tmSummer.getTimezoneOffset();
	tmWinter = new Date(Date.UTC(2005, 12, 30, 0, 0, 0, 0));
	wo = -1 * tmWinter.getTimezoneOffset();

	if (-660 == so && -660 == wo) return 'Pacific/Midway';
	if (-600 == so && -600 == wo) return 'Pacific/Tahiti';
	if (-570 == so && -570 == wo) return 'Pacific/Marquesas';
	if (-540 == so && -600 == wo) return 'America/Adak';
	if (-540 == so && -540 == wo) return 'Pacific/Gambier';
	if (-480 == so && -540 == wo) return 'US/Alaska';
	if (-480 == so && -480 == wo) return 'Pacific/Pitcairn';
	if (-420 == so && -480 == wo) return 'US/Pacific';
	if (-420 == so && -420 == wo) return 'US/Arizona';
	if (-360 == so && -420 == wo) return 'US/Mountain';
	if (-360 == so && -360 == wo) return 'America/Guatemala';
	if (-360 == so && -300 == wo) return 'Pacific/Easter';
	if (-300 == so && -360 == wo) return 'US/Central';
	if (-300 == so && -300 == wo) return 'America/Bogota';
	if (-240 == so && -300 == wo) return 'US/Eastern';
	if (-240 == so && -240 == wo) return 'America/Caracas';
	if (-240 == so && -180 == wo) return 'America/Santiago';
	if (-180 == so && -240 == wo) return 'Canada/Atlantic';
	if (-180 == so && -180 == wo) return 'America/Montevideo';
	if (-180 == so && -120 == wo) return 'America/Sao_Paulo';
	if (-150 == so && -210 == wo) return 'America/St_Johns';
	if (-120 == so && -180 == wo) return 'America/Godthab';
	if (-120 == so && -120 == wo) return 'America/Noronha';
	if (-60 == so && -60 == wo) return 'Atlantic/Cape_Verde';
	if (0 == so && -60 == wo) return 'Atlantic/Azores';
	if (0 == so && 0 == wo) return 'Africa/Casablanca';
	if (60 == so && 0 == wo) return 'Europe/London';
	if (60 == so && 60 == wo) return 'Africa/Algiers';
	if (60 == so && 120 == wo) return 'Africa/Windhoek';
	if (120 == so && 60 == wo) return 'Europe/Amsterdam';
	if (120 == so && 120 == wo) return 'Africa/Harare';
	if (180 == so && 120 == wo) return 'Europe/Athens';
	if (180 == so && 180 == wo) return 'Africa/Nairobi';
	if (240 == so && 180 == wo) return 'Europe/Moscow';
	if (240 == so && 240 == wo) return 'Asia/Dubai';
	if (270 == so && 210 == wo) return 'Asia/Tehran';
	if (270 == so && 270 == wo) return 'Asia/Kabul';
	if (300 == so && 240 == wo) return 'Asia/Baku';
	if (300 == so && 300 == wo) return 'Asia/Karachi';
	if (330 == so && 330 == wo) return 'Asia/Calcutta';
	if (345 == so && 345 == wo) return 'Asia/Katmandu';
	if (360 == so && 300 == wo) return 'Asia/Yekaterinburg';
	if (360 == so && 360 == wo) return 'Asia/Colombo';
	if (390 == so && 390 == wo) return 'Asia/Rangoon';
	if (420 == so && 360 == wo) return 'Asia/Almaty';
	if (420 == so && 420 == wo) return 'Asia/Bangkok';
	if (480 == so && 420 == wo) return 'Asia/Krasnoyarsk';
	if (480 == so && 480 == wo) return 'Australia/Perth';
	if (540 == so && 480 == wo) return 'Asia/Irkutsk';
	if (540 == so && 540 == wo) return 'Asia/Tokyo';
	if (570 == so && 570 == wo) return 'Australia/Darwin';
	if (570 == so && 630 == wo) return 'Australia/Adelaide';
	if (600 == so && 540 == wo) return 'Asia/Yakutsk';
	if (600 == so && 600 == wo) return 'Australia/Brisbane';
	if (600 == so && 660 == wo) return 'Australia/Sydney';
	if (630 == so && 660 == wo) return 'Australia/Lord_Howe';
	if (660 == so && 600 == wo) return 'Asia/Vladivostok';
	if (660 == so && 660 == wo) return 'Pacific/Guadalcanal';
	if (690 == so && 690 == wo) return 'Pacific/Norfolk';
	if (720 == so && 660 == wo) return 'Asia/Magadan';
	if (720 == so && 720 == wo) return 'Pacific/Fiji';
	if (720 == so && 780 == wo) return 'Pacific/Auckland';
	if (765 == so && 825 == wo) return 'Pacific/Chatham';
	if (780 == so && 780 == wo) return 'Pacific/Enderbury'
	if (840 == so && 840 == wo) return 'Pacific/Kiritimati';
	return 'US/Pacific';
}

August 30, 2007

War Story: Switching from Movable Type to WordPress MU

As some of you may have noticed, we made the switch from Movable Type (v3.33) to WordPress MU on July 9th. We thought it would be a great idea to discuss why we made the switch as well as the technical requirements of making the switch. We’re currently serving seven blogs, containing 2,034 posts by 47 authors, with 1,924 comments. The switch took roughly 60 work hours including research, and the effort was led by me, an intern, with our product and ops teams supervising.

The most compelling reason for us to switch was WordPress’s usability — managing WordPress is strait up easier from a source code, data, and content management perspective (authoring, user management, etc). There were many specific reasons for the switch as well. First, WordPress, with all of its plugins, can offer an unbelievable amount of functionality. For example, with WordPress we could offer features such as author profiles and archive paging functionality. Our previous installation of Movable Type was also somewhat chaotic, making maintenance and expansion grueling. Find below a feature comparison between Movable Type and WordPress MU.

Keep in mind that the list of features below does not perfectly represent each platform’s feature set. The list instead contains features that we at Redfin wanted for our blog.
wp_table.png

Though this chart suggests that WordPress is the better choice for us, we had to consider the side effects of switching, the largest side effect being older permalinks potentially being broken. Other side effects were a potential loss of data (posts, comments, authors), a requirement for more blogger training, and having to familiarize ourselves with a new platform. Once we decided that we were willing to deal with these side effects, we began the migration process.

clip_image001.jpg

Both Movable Type and WordPress MU offer importing and exporting functionality, which made the task of getting our data into WordPress very simple. The WordPress import function also requires that all authors be in the system before the import begins, so we added our authors before performing the import. The reason for this is because the WordPress import process has a step that forces the importer to make author correlations between the imported data and the WordPress author list. The author correlation step provides a list of authors that were found in the imported data and requests that the importer decide how these authors correlate to the authors in WordPress. Unfortunately, this feature was broken in our installation of WordPress, so we were forced to write a custom script to correct the WordPress MU database by using the old Movable Type database.

With our data in place, the next largest task was to ensure that our old permalinks be properly redirected. To do this, we wrote a script that used the Movable Type database to produce a large set of Apache rewrite rules that redirect old permalink requests. To limit the number of required rewrite rules, we configured our WordPress permalink structure to match Movable Type’s (/year/month/post_title_etc.html). This also required an additional WordPress plugin that forces permalinks to use underscores instead of the default hyphens.

clip_image002.jpg

The last hurdle we ran into during this migration process was achieving our desired link structure. WordPress MU allows blogs to either appear as sub domains (new_blog.domain.com) or folders (domain.com/new_blog). We wanted our blogs to be accessible by blog.redfin.com, sfbay.redfin.com/blog, and seattle.redfin.com/blog, etc, and after lots of fiddling with WordPress, we concluded that we could not use the sub domain option. To get around this, we configured WordPress to use folders (blog.redfin.com/sfbay, blog.redfin.com/seattle, etc). We then configured a virtual host for each blog, sfbay.redfin.com, seattle.redfin.com, etc, to reverse proxy the /blog folder of these virtual hosts to the blog.redfin.com/sfbay, blog.redfin.com/seattle, etc folders. The reverse proxy article showed us the road, but the path we took was slightly different. Instead of using ProxyPass and ProxyPassReverse, we used the [P] option in a RewriteRule. This allowed us to apply other rewrite rules to the /blog folder, where the ProxyPass and ProxyPassReverse option did not.

We’re very happy with WordPress MU. In fact, we’re happier with WordPress MU than we were with Movable Type. The migration process was not easy, but it was worth the benefits of switching to WordPress. Feel free to post some comments if you’d like more specifics about the switch.