Yesterday one of my co-workers spent hours reproducing and tracking down a bug that turned out to be a stray comma.
Unfortunately, Microsoft Internet Explorer doesn’t support trailing commas in JavaScript arrays and object literals. Code like this won’t parse:
var chord = ["do", "mi", "so",];
var json = { truth:"beauty", beauty:true, };
Those final commas will cause syntax errors in IE.
“How many thousands of developer hours have been lost to random IE bugs like this?” I asked myself. I decided that there had to be a good way to detect this problem in an automated way, without firing up a copy of IE and running a full test suite.
It turns out that these and other syntax errors can be detected automatically from the Windows command line, using the Windows Scripting Host (WSH). On Windows XP and higher, the command-line tool “cscript.exe” can be used to run JavaScript (ahem, JScript) headlessly (outside of any browser).
Just create a file called “wsh-parser.js” like this:
var fso = new ActiveXObject( "Scripting.FileSystemObject" );
function parse(fname) {
var file = fso.OpenTextFile( fname, 1 );
ret = file.ReadAll();
file.Close();
try {
eval("(function(){\n"+ret+"\n});");
} catch (e) {
WScript.Echo("Syntax error parsing " + fname);
WScript.Echo(" " + e.message);
}
}
function findJavaScript(folder) {
for (var fc = new Enumerator(folder.files); !fc.atEnd(); fc.moveNext()) {
var file=fc.item();
if (/.js$/.test(file.Name)) {
parse(file);
}
}
for (var fc = new Enumerator(folder.subfolders); !fc.atEnd(); fc.moveNext()) {
var subfolder = fc.item();
if (subfolder.Name == ".svn") continue; // ignore .svn folders
findJavaScript(subfolder);
}
}
var rootPath = "src/main/javascript";
var rootFolder = fso.GetFolder(rootPath);
findJavaScript(rootFolder);
Then run it like this:
cscript //E:javascript //nologo wsh-parser.js
The script will automatically examine every JavaScript file in the specified rootPath, reporting errors in any file that won’t parse correctly.
There’s a bit of magic going on here in the “parse” function to guarantee that we’re only verifying the syntax of our JavaScript, without actually running it. The magic line is:
eval("(function(){\n"+ret+"\n});");
The “eval” command would normally execute the provided string, but here we wrap the code in an anonymous function declaration. We improve the performance and the maintainability of our test by declaring the function without actually using it.
Note that you can also use JsLint to detect trailing commas, but it will probably report a bunch of other problems with your code that you may or may not care about.
Automated validators like these should be used together with headless unit tests as well as live browser tests. (I would recommend Selenium for automated browser tests. There are a ton of excellent headless JavaScript unit test tools, including Schaible’s JsUnit, RhinoUnit, and DOH. Note that Hieatt’s JsUnit is not headless.)
But I recommend running automated validators first, because they’re fast and the errors they report are easy to debug/fix. Once the automated validators are happy, run your unit tests. When your unit tests pass, run your live browser tests.
I recently heard that some loyal Redfin customers were using Google to do address searches. That’s a shame, since Redfin does a pretty decent job of searching for addresses, MLS IDs, cities, etc. I wanted to see if I could help those power users get to Redfin search results with fewer clicks.
Naturally, I wanted to see if I could make a Redfin search available. It was surprisingly easy- I didn’t have to change any Redfin code. I just wrote an XML file, hosted it on a Web server, and put a link into an HTML page (this page, in fact!)
If you’re using Firefox or Internet Explorer 7, you should be able to enable Redfin search by choosing the relevant option in the search dropdown while viewing this page. Here’s a screenshot of Firefox:
Here’s how it looks in IE7:
Once you’ve added Redfin, you can do address searches on Redfin using the search control:
Another way to do searches in Redfin is to use a keyword bookmark. In Firefox, make a new bookmark, and edit it to look like this:
The important parts are the location and the keyword. The location got cut off in my screenshot, but it should be http://www.redfin.com/stingray/do/listings-search#search_location=%s
(the only tricky part is the “%s” part, which will get replaced with whatever you search for.)
To use it, you type the keyword and the search terms into your location bar, like this:
One of the main goals of our latest release was to improve the overall performance of the user interface, so we’re starting an intermittent series today on the dev blog talking about what we learned along the way.
Firstly, we had to recognize (channeling Steve Souders of YSlow and High Performance Web Sites fame) that the largest part of our performance problems were on the client-side, and the problems were most severe in IE6.
We set about trying to optimize client performance independent of network latencies and server query times. For the most part, this meant reducing the time the browser spent running our JavaScript. One reason this is particularly important is that the browser does not do anything else while JavaScript is running; the UI is completely locked up. No events, no back button, no browser menus.
If you do a lot of heavy processing on the client side in JavaScript, this can be a real problem since it causes visible delays and makes a web application generally clunky and unresponsive. Accepting that there were times when our website was sluggish, we set out to improve the overall performance of our user interface. More about how we figured out where to start after the jump. Read the rest of this entry »
I arrived at work today with a perplexing tech support e-mail in my inbox. A customer was claiming that although Redfin had never worked well in Firefox, last week we totally broke it. Needless to say, we were all a little surprised (and more than a little worried) over here, since we use Firefox all day every day. When we asked the user to send us screenshots, they looked something like this:
This is when we got well and truly confused. It looked like the browser was downloading our CSS, because items were appearing in the right place on the page, but all color had drained out of the world. Was Firefox in some sort of a funk? At first, we guessed that perhaps the user had some sort of ad blocker that was blocking images from our content delivery network, but we quickly noticed that some images made it through. What was going on here? Solution after the jump.
UPDATED: As of January 31, 2008, Redfin supports Safari 3 without any of the trickery described below. We’ll leave the post here for the curious, but in the meantime, there’s no reason to go through these steps on our account.
Last January, we ported Redfin from a proprietary map platform to Microsoft’s Virtual Earth, a move that has enabled us to move faster in adding new markets and features. We’ve gotten both praise and criticism for the move over the last 9 months, but no piece of feedback has been as constant or as vociferous as the cry of Mac users who want us to support Safari.
As a MacBook Pro user myself, I understand the desire for us to support the native Mac browser (of course, we do support Firefox on OS X), but Virtual Earth didn’t support Safari, so we were pretty much out of luck. And although I like Safari, we have to admit that Safari 2 is buggy at best with support for CSS and JavaScript. Applications like Redfin or Virtual Earth that use a lot of client-side scripting are bound to have problems with Safari 2. This doesn’t mean that it’s impossible to support Safari 2, but it is likely to require significant testing and engineering to get a site to work properly.
The upcoming Safari 3, though, has taken a massive leap forward in terms of standards support, and it’s crazy fast to boot. While we don’t support Safari yet officially, you can use Safari 3 with Redfin if you trick Redfin into thinking that you are running Firefox, and it Mostly Works ™.
Before you start, please note that Safari 3 is in beta, it is not to be used on mission-critical systems, this is all at your own risk, your computer could spontaneously explode, etc. With that disclaimer out of the way, here’s how you get Safari and Redfin to play nice:
Download a copy of Safari 3, either the Safari 3 Public Beta from Apple or the WebKit nightly from webkit.org. The Safari Public Beta is a better tested build, but it may not have all the bug fixes of the nightly build from webkit.org. Also note that installing the Safari 3 Public Beta will get rid of Safari 2 on OS X, whereas I believe (but am not 100% sure) that the WebKit nightly can co-exist with Safari 2.
Turn on debug functionality for Safari. On Macs, do it this way:
Quit Safari so it’s not running.
Run the program called “Terminal” which is located in your Applications->Utilities folder.
Type the following command into Terminal exactly as shown:
Quit from the Terminal program, and launch Safari again. Safari should now have a “Debug” menu. If not, check the command you typed above. Case is important.
On Windows boxes, turn on the debug menu this way:
Quit Safari so it’s not running.
Open the file at:
c:Documents and Settingsyour usernameApplication Data
Apple ComputerSafariPreferences.plist
Add the following text to the file before the final </dict>:
<key>IncludeDebugMenu</key>
<true/>
Launch Safari again. Safari should now have a “Debug” menu.
Launch Safari 3. In the Debug menu, select User-Agent –> Firefox 2.0.x.x. (The exact version of Firefox in the Debug menu depends on which build of Safari you have.)
Go to www.redfin.com, and use Redfin to your heart’s content.
There are a few things to note here. First, you will have to switch the User Agent every time you open a new tab or restart Safari. Second, there are also a few known bugs we’ve seen with just the limited testing we’ve done:
Depending on which version of Safari 3 you get, the images of houses next to the map may be stretched weirdly.
The zoom bar on the map isn’t centered correctly.
Keyboard shortcuts on the map page (up/down/left/right) don’t work correctly.
If you try this out and find any other bugs (and we’re sure they’re there), let us know in the comments section.
As was mentioned a fewplaces on the web today, we’re proud to announce that Redfin has contributed to a project that will open source TurboGrid, the excellent JavaScript grid control made by the TurboAjax group.
We’ve been really trying to hammer on our map page performance for the last few weeks, and it became clear early on that performance of our current grid is suffering when there are a lot of houses on the map. As with many of our performance problems, it is especially pronounced in Internet Explorer 6. (Sidenote: Glenn recently told a focus group of customers that I would personally pay them $20 each to upgrade from Internet Explorer 6 or switch to Firefox, and I’m beginning to come around to the idea despite the current state of my bank account.)
The most frustrating part for me personally is that the solution for slow performing list controls is fairly well understood and exists in practicallyeverymajorwindowlibrary: virtual list controls. A virtual list control is one in which you don’t add all the items to the list immediately on creation but rather hand the list control a callback function instead. The list then only asks for the handful of rows it needs to display at any one moment and can therefore scale up to thousands or millions of rows. In pseudo-code, that looks something like this:
function initList() {
var list = new VirtualList();
list.setRowCount(1000);
list.setRowContent(contentCallback);
}
function contentCallback(rowIndex) {
return "this is the content for row #" + rowIndex;
}
TurboGrid uses the virtual list strategy, and it reacts much more quickly than our current grid control. Our next release will use TurboGrid to speed up the list, and we’ve been very pleased with both its performance and its feature set. So why am I frustrated? Well, it’s mostly because I learned about virtual lists in my first few months of professional coding after college in (ahem) 1997. And while browser-based AJAX development is a lot of fun, it can be disheartening when you are re-solving problems that you thought were solved over a decade ago.
In the meantime, though, we’re happy we found out about the TurboGrid open source project from the good folks at Sitepen and were able to help in some small way to make it a reality. Until we all have jetpacks, flying cars, and a built-in grid control in HTML, TurboGrid will help keep us chugging along, and we hope that users will notice the speed up (although that $20 is still on the table for IE6 users).
Writing rich date/time features in a web app can be a pain. Apps (such as schedulers) that do math on times (e.g. ordering times) should pay attention to time zones for those times, but it’s difficult to know which time zone should be used to display the times to the user. Asking the user to explicitly choose a time zone is natural and often necessary, but a long list of time zone choices can be intimidating to the user. I’ll discuss one method you can use to detect the probable time zone of a browser. It’s not perfect, but it offers a good default (and it’s easy to code.)
For apps that don’t have rich date/time functionality, times can be represented as simple numbers or strings. For example, if I wanted to meet you in San Francisco to go to the late showing of The Bourne Ultimatum at the Metreon theater, it’s probaby fine to say “let’s meet at the theater at 11:15 PM on Friday; the show starts at 11:30 PM.” Since we’re in the same time zone as each other, and as the theater, we don’t really care about time zones. An application that’s facilitating this interaction could store and display the times “11:15 PM” and “11:30 PM” without regard to time zones.
If we want to do date/time processing on the back end (e.g. ordering events in time), it’s more general to store absolute times in the database. For example, instead of storing something like “hour => 11, minute => 15″, which might mean different absolute times to different users, it is convenient to store a canonical time such as the number of seconds since January 1, 1970 in GMT. That way, we can compare all of the times in our database without having to worry about the time zone for each.
If we store “absolute” times in the database, then displaying times to users becomes a localization issue. It’s pretty similar to localizing web site content into the language of the user. You start with a notion of what you want to present to the user, then you identify how the user wants to receive it (i.e. their language/locale), and then you localize the content at runtime (i.e. you show them the content in their language.)
As with language, you need to know about the user. The HTTP protocol specifies that the “Accept-Language” header can be used by servers to find out which language(s) the user prefers. The “Accept-Language” header is nice because it lets websites show content in the “correct” language without having to explicitly ask the user. A user who only speaks French doesn’t have to puzzle through an English language page that says “click here for French” somewhere in a footer- they just see the content in French. Even better, it’s one less setting that the server has to manage, and that the user has to set and keep up-to-date.
Unfortunately, there is no corresponding “Accept-Timezone” header- the HTTP standard does not contain any facility to allow the browser to automatically tell the server what time zone the user cares about.
There are two standard ways for developers to deal with this.
First, they can ignore it. For many apps, this is a decent approach- just store “11:30 PM”, and don’t worry about the time zone. As long as all the users who care about that time know what time zone it’s in, then the app doesn’t have to keep track of it.
Second, they can ask the user to make an explicit choice. For example, when setting up Google Calendar, you are asked to choose a time zone. That’s fine for the developer, but finding the “right” time zone in a long list can be a pain for the user.
I wanted to let users choose a time zone on my site, but I also wanted to have an intelligent default- for most users, they shouldn’t have to take any action; the choice I make for them should be correct.
This calls for Javascript on the client. I wanted to write some Javascript that would choose the right option in a time zone dropdown.
This is slightly harder than it seems because Javascript ALSO does not contain a way to get the time zone of the user. Javascript DOES, however, provide a way to get the offset from GMT for any particular time. A time zone can be thought of as a rule that says what the GMT offset is for different times. We can therefore do a reverse mapping- if we know the GMT offset for a few times, we can figure out the time zone for the user. Time zones can be quite complicated (some include Daylight Savings Time, some start or end DST on different dates than others, sometimes the DST offset isn’t a full hour, etc. There are even time zones that are identical for all FUTURE times, but had differences in the PAST.)
In theory, we could deal with all of these cases by doing many probes- we could check the GMT offset for many times, and get an exact time zone match. In practice, this really isn’t necessary- most users are in the more populous time zones, and the cost of failure (defaulting to a time zone that’s similar but not quite right) is not terribly high. Instead, we can probe two times (one in the summer and one in the winter) to find out the normal GMT offset, whether the time zone has Daylight Savings Time, and the DST offset.
In terms of implementation, I wanted to basically make a list of recognized offsets. That is, a list that says “if the summer offset is -7 hours, and the winter offset is -8 hours, then the time zone is probably US/Pacific.”
I like hacking in Ruby, so I grabbed the TZInfo Ruby library, and wrote some code to run through the known time zones, figuring out the winter and summer offsets for each. After grouping by offsets, I had to choose a winner in the case of duplicates. When multiple time zones had the same summer and winter offsets, I searched for each of them on Google. I figured that the time zone with the most hits was probably the most popular one, so I chose that one. Here’s the Javascript code that I came up with:
function getTimezoneName() {
tmSummer = new Date(Date.UTC(2005, 6, 30, 0, 0, 0, 0));
so = -1 * tmSummer.getTimezoneOffset();
tmWinter = new Date(Date.UTC(2005, 12, 30, 0, 0, 0, 0));
wo = -1 * tmWinter.getTimezoneOffset();
if (-660 == so && -660 == wo) return 'Pacific/Midway';
if (-600 == so && -600 == wo) return 'Pacific/Tahiti';
if (-570 == so && -570 == wo) return 'Pacific/Marquesas';
if (-540 == so && -600 == wo) return 'America/Adak';
if (-540 == so && -540 == wo) return 'Pacific/Gambier';
if (-480 == so && -540 == wo) return 'US/Alaska';
if (-480 == so && -480 == wo) return 'Pacific/Pitcairn';
if (-420 == so && -480 == wo) return 'US/Pacific';
if (-420 == so && -420 == wo) return 'US/Arizona';
if (-360 == so && -420 == wo) return 'US/Mountain';
if (-360 == so && -360 == wo) return 'America/Guatemala';
if (-360 == so && -300 == wo) return 'Pacific/Easter';
if (-300 == so && -360 == wo) return 'US/Central';
if (-300 == so && -300 == wo) return 'America/Bogota';
if (-240 == so && -300 == wo) return 'US/Eastern';
if (-240 == so && -240 == wo) return 'America/Caracas';
if (-240 == so && -180 == wo) return 'America/Santiago';
if (-180 == so && -240 == wo) return 'Canada/Atlantic';
if (-180 == so && -180 == wo) return 'America/Montevideo';
if (-180 == so && -120 == wo) return 'America/Sao_Paulo';
if (-150 == so && -210 == wo) return 'America/St_Johns';
if (-120 == so && -180 == wo) return 'America/Godthab';
if (-120 == so && -120 == wo) return 'America/Noronha';
if (-60 == so && -60 == wo) return 'Atlantic/Cape_Verde';
if (0 == so && -60 == wo) return 'Atlantic/Azores';
if (0 == so && 0 == wo) return 'Africa/Casablanca';
if (60 == so && 0 == wo) return 'Europe/London';
if (60 == so && 60 == wo) return 'Africa/Algiers';
if (60 == so && 120 == wo) return 'Africa/Windhoek';
if (120 == so && 60 == wo) return 'Europe/Amsterdam';
if (120 == so && 120 == wo) return 'Africa/Harare';
if (180 == so && 120 == wo) return 'Europe/Athens';
if (180 == so && 180 == wo) return 'Africa/Nairobi';
if (240 == so && 180 == wo) return 'Europe/Moscow';
if (240 == so && 240 == wo) return 'Asia/Dubai';
if (270 == so && 210 == wo) return 'Asia/Tehran';
if (270 == so && 270 == wo) return 'Asia/Kabul';
if (300 == so && 240 == wo) return 'Asia/Baku';
if (300 == so && 300 == wo) return 'Asia/Karachi';
if (330 == so && 330 == wo) return 'Asia/Calcutta';
if (345 == so && 345 == wo) return 'Asia/Katmandu';
if (360 == so && 300 == wo) return 'Asia/Yekaterinburg';
if (360 == so && 360 == wo) return 'Asia/Colombo';
if (390 == so && 390 == wo) return 'Asia/Rangoon';
if (420 == so && 360 == wo) return 'Asia/Almaty';
if (420 == so && 420 == wo) return 'Asia/Bangkok';
if (480 == so && 420 == wo) return 'Asia/Krasnoyarsk';
if (480 == so && 480 == wo) return 'Australia/Perth';
if (540 == so && 480 == wo) return 'Asia/Irkutsk';
if (540 == so && 540 == wo) return 'Asia/Tokyo';
if (570 == so && 570 == wo) return 'Australia/Darwin';
if (570 == so && 630 == wo) return 'Australia/Adelaide';
if (600 == so && 540 == wo) return 'Asia/Yakutsk';
if (600 == so && 600 == wo) return 'Australia/Brisbane';
if (600 == so && 660 == wo) return 'Australia/Sydney';
if (630 == so && 660 == wo) return 'Australia/Lord_Howe';
if (660 == so && 600 == wo) return 'Asia/Vladivostok';
if (660 == so && 660 == wo) return 'Pacific/Guadalcanal';
if (690 == so && 690 == wo) return 'Pacific/Norfolk';
if (720 == so && 660 == wo) return 'Asia/Magadan';
if (720 == so && 720 == wo) return 'Pacific/Fiji';
if (720 == so && 780 == wo) return 'Pacific/Auckland';
if (765 == so && 825 == wo) return 'Pacific/Chatham';
if (780 == so && 780 == wo) return 'Pacific/Enderbury'
if (840 == so && 840 == wo) return 'Pacific/Kiritimati';
return 'US/Pacific';
}
