Where’s My Jetpack?

JetpackAs was mentioned a few places on the web today, we’re proud to announce that Redfin has contributed to a project that will open source TurboGrid, the excellent JavaScript grid control made by the TurboAjax group.

We’ve been really trying to hammer on our map page performance for the last few weeks, and it became clear early on that performance of our current grid is suffering when there are a lot of houses on the map. As with many of our performance problems, it is especially pronounced in Internet Explorer 6. (Sidenote: Glenn recently told a focus group of customers that I would personally pay them $20 each to upgrade from Internet Explorer 6 or switch to Firefox, and I’m beginning to come around to the idea despite the current state of my bank account.)

The most frustrating part for me personally is that the solution for slow performing list controls is fairly well understood and exists in practically every major window library: virtual list controls. A virtual list control is one in which you don’t add all the items to the list immediately on creation but rather hand the list control a callback function instead. The list then only asks for the handful of rows it needs to display at any one moment and can therefore scale up to thousands or millions of rows. In pseudo-code, that looks something like this:


function initList() {
var list = new VirtualList();
list.setRowCount(1000);
list.setRowContent(contentCallback);
}


function contentCallback(rowIndex) {
return "this is the content for row #" + rowIndex;
}

TurboGrid uses the virtual list strategy, and it reacts much more quickly than our current grid control. Our next release will use TurboGrid to speed up the list, and we’ve been very pleased with both its performance and its feature set. So why am I frustrated? Well, it’s mostly because I learned about virtual lists in my first few months of professional coding after college in (ahem) 1997. And while browser-based AJAX development is a lot of fun, it can be disheartening when you are re-solving problems that you thought were solved over a decade ago.

In the meantime, though, we’re happy we found out about the TurboGrid open source project from the good folks at Sitepen and were able to help in some small way to make it a reality. Until we all have jetpacks, flying cars, and a built-in grid control in HTML, TurboGrid will help keep us chugging along, and we hope that users will notice the speed up (although that $20 is still on the table for IE6 users).

(Photo credit: samjuk on Flickr; Title credit: Daniel H. Wilson’s Where’s My Jetpack?)

Discussion

  • Dutch Marlowe

    Two alternative grid controls with AJAX loading via a callback function that can then use the “OFFSET” parameters in whatever flavor of SQL to scroll through the cursor on the server and get the appropriate subset of records. Yes, the query is re-executed each time, but hey, that’s a server problem and I haven’t seen an AJAX control that managed to keep a cursor open while loosely connected.

    http://www.scbr.com/docs/products/dhtmlxGrid/ – if you are not into additional libraries
    http://www.extjs.com – with adapters for YUI + jQuery (I think also Dojo because I see you are OK with Dojo).

    There are probably others. An intersting thing is that at least with the extjs grid you can have the server return XML or JSON (very nice with Ruby/Rails), have the records correctly packaged, and displayed in the grid with literally one client URL request and one call on the server side.

  • Sam Johnson

    Thanks for the credit, it’s so nice when people take the time to do things properly :)

    interesting bit of code

    Sammy J

  • http://www.redfin.com Sasha

    Thanks for the comment, “Dutch”. Glad to see you around these parts.

    Interesting point about a server-side pagination triggered by a client-side list. TurboGrid could in theory do this kind of pagination as well, although the idea of keeping cursors open scares me, frankly.

    In addition, since we don’t paginate on the map, it doesn’t make as much sense for us to have the grid use a callback that goes to the server when the user scrolls. We have to send a fair amount of data down about each home just in order to show it on the map (where it is, beds, baths, etc.). At that point, we might as well send along the extra fields that the grid needs.

    Finally, in our experience the latencies involved in having a user wait around while an AJAX call is out are not good. In previous versions, we would fetch info about a house for the sidepane next to the map once a user clicked a house, and the latency that built in to the UI was extremely annoying for users. In our current dev builds, we’ve spent a lot of time lowering the amount of time that AJAX requests hold up the UI.

    As for Ext, it seems to be a pretty good library, and it’s built on YUI, Prototype/Scriptaculous, or comes with its own control library. It doesn’t build on Dojo, although I think it can be used simultaneously with it.

    As for the one url/one server-side call, that’s great for a lot of situations, but it’s not so great for us, as we need to distribute the information from the AJAX call to the map, the grid, and the sidepane. We don’t want the grid to do the AJAX calls because we need to use the results in several controls.

    Thanks again for the thoughtful comment.

  • http://www.attributor.com Adrian McDermott

    Sasha,
    will you be working on a lotus notes adapter for redfin next?

    –adrian

  • http://devblog.redfin.com/author/sasha.aickin Sasha

    Adrian,

    I’m halfway through the Lotus Notes adapter; it’s going to be awwwwwwesome. Each house is going to be associated with a Notes database, and you can add unstructured content to every house through custom-built Domino forms.

    Sasha.

    P.S. For folks who don’t know Adrian, he’s the guy who originally taught me about virtual list controls back in 1997. We were working at a company called Plumtree Software, and one of the early projects I worked on there was a Lotus Notes adapter for Plumtree’s system. It was, and still remains, the most fly-by-the-seat-of-my-pants software project I’ve ever worked on.

  • http://iligan.nu Stasigr

    Hello, very nice site, keep up good job!
    Admin good, very good.