Internet Explorer 9 Beta sometimes gives the wrong answer when we ask for the size of the viewport (viewable area of the browser window).  On an HTML document with “X-UA compatible” set to IE=7 and Windows font DPI set to 125% or 150% with the browser window maximized, IE9 claims the viewport is a few pixels larger than it actually is.

Soon after Microsoft released IE9 Beta, we began receiving numerous tech support e-mails regarding our map page being stuck in an endless loop where the map continually refreshes.  None of our engineers were able to reproduce this problem, so we classified this as a non-issue because the problem only occurred in IE9 beta, which is not an official release.  But the tech support e-mails kept coming in, so we decided to find out exactly what was going on.  We did not know if it was a problem in our code or a bug on Microsoft’s part.

To get a better understanding of the situation, we turned to our troubled users for help.  Our first lead came from Nitin G. who mentioned that setting the display font very small cured his problem.  This helped, but we needed more information.  Mike P., another Redfin user, noticed that the problem only occurred when the browser window is maximized.  He also reported seeing scroll bars appear and disappear during the infinite loop, and the loop stopping when he opened developer tools (F12).  Not only did he provide us with screenshots, but he sent us a video capture of the bug as it was happening.  Here are a couple of the screenshots:

(Figure 1)

(Figure 2)

The first thing we wanted to do was to match his exact search and display.  We tried changing the display resolution, but with no luck.  Then we remembered Nitin’s comment about shrinking the font size, and it was clear what we needed to do to reproduce the problem.  Change the font DPI settings in Windows!

We successfully reproduced the problem by setting Window’s display text DPI to a percentage larger than standard and maximizing the window.  By simply setting the DPI to 125% or 150%, this bug caused scroll bars to appear (Figure 1) and disappear (Figure 2) on the map page, causing our map to resize, which in-turn executes a new search.  Scroll bars only appeared during a search, not when a new search is kicked-off.   So now we have a reason for the infinite loop, but what is the underlying cause?  It turns out that during a search, we lock our UI by creating an HTML element spanning the entire browser’s viewport, preventing users from searching the map as listings are being populated.  It is when we lock our UI that the scroll bars appear, and this only happens when “X-UA compatible” is set to IE=7 in the HTML document.

We use Dojo extensively in our site, and one of its uses is to obtain the browser’s viewport size by calling dojo.window.getBox(), which contains the viewport’s height and width.  These properties return values larger than the actual size of the viewport when the browser window is maximized, which cause scroll bars to appear when we call on the UI locker.  We created a reduced test case where we used dojo.window.getBox() to make a <div> that spans the entire browser viewport:

Sure enough, we reproduced the problem.  This led us to believe that this is a bug with Dojo, but we needed to be certain.  Digging deeper into Dojo’s code, dojo.window.getBox() uses document.documentElement and calls on the element’s clientHeight and clientWidth properties to get the viewport dimensions.  We created another, similar reduced test case, but without Dojo:

The problem was still there!  This confirms that there is a bug with IE9’s viewport code, and not Dojo.  Although the problem is not with Dojo, we fixed this bug by modifying Dojo’s method for returning the viewport because Dojo calls on the DOM for the dimensions.  Specifically, we subtract a few pixels from the height and width.

Special thanks goes to Nitin G. and Mike P. for helping us solve the mysterious IE9 infinite loop bug.  Without their help and the help of our other avid Redfin users, this bug would have gone unnoticed.  We use and consider many feedbacks and tech support problems to improve our site time and time again.  We appreciate everyone’s help.

However when that content is cached, you want to serve it up as quickly as possible. If you’ve already calculated the content, you’d like to include it inline in the page, without requiring an AJAX roundtrip. That way, you avoid the latency of an unnecessary round-trip. You also allow the page to be fully rendered (so content doesn’t jump around), etc.

You can optimize for the empty cache, or you can optimize for the full cache, but it seems hard to optimize both experiences.

Redfin faces exactly this conundrum with our listing pages (e.g. http://www.redfin.com/CA/San-Francisco/830-El-Camino-Del-Mar-94121/home/604622.) Calculating the Similar Listings and Similar Sales is expensive and performed in real time. We cut this Gordian Knot through the use of the Varnish caching reverse proxy, along with clever use of ESI (Edge Side Includes.) For an overview of how we use Varnish at Redfin, see our previous post.

We want to say “if there’s a cache miss, then do AJAX, but if there’s a cache hit, then just include the content.” We have to make sure that the AJAX calls will fill the cache, such that subsequent requests will see cache hits, of course!

I’ll outline what the requests/responses look like for us, then I’ll include some pseudocode that supports this.

At the beginning of time, the cache is empty, and the browser requests information on a Listing.

That’s all great, but we still haven’t used the cache! The cache entry will get used for subsequent requests for the same page, like this:

There are two things worth noting about this exchange.

First, when the backend server gets a request for /similars?property_id=604622, it has to decide if it should be returning the real HTML, or should be returning Javascript that will retrieve the HTML via AJAX. It makes this decision based on the value of a header passed in by the client. When the client is making an AJAX request, it knows it better NOT get back a response that generates AJAX requests (that’d be a death spiral.) Therefore, when it makes the AJAX request, it includes the special header. In all other cases, the special header is NOT included. When the header is included in a request, the server will generate the real HTML. When the header is not included, Varnish may answer the request from cache, or it may pass through to the backend server. If the request is fulfilled by the Varnish cache, then it’s the real HTML, but if it’s fulfilled by the backend server, it’ll be the AJAXy HTML.

Second, there are two URLs that have to do with similars.

/similars?property_id=604622 is an internal-use-only URL that returns the content (either the proper HTML or the AJAX code.)

/extranet-similars?property_id=604622 is an externally facing URL that only returns an ESI fragment (which will subsequently be filled in by Varnish. This way, the ESI endpoints are never available to the extranet; Varnish can get to them, but extranet clients have no need for them. This lets us be lazy with the ESI URLs. For example, URLs that are exposed to the extranet do extra validation to check if the user is logged in, etc. URLs for internal use only, such as the ESI URLs, can skip that work. This also lets us change the URLs when the property changes, to facilitate cache busting (see the “Cache busting” section in ESI and Caching Trickery in Varnish for more information.

Pseudocode

OK, so we know what we want the interaction to look like. What code will make this happen? Here’s some Javaish pseudocode that illustrates how it might work:


/*
Invoked for requests like http://www.redfin.com/[address]/home/[property id]
*/
public void handlePropertyRequest(Request request, Response response, long propId) {
   Property property = getProperty(propId);
   response.write("<html><head></head><body>" +
      ...
      "<esi:include src='/extranet-similars?property_id=" +
         propId +
         "&last_mod=" +
         property.getLastModified() +
      "'/>" +
      ...
      "</body></html>");
}



/*
Invoked for (extranet) requests like /extranet-similars?property_id=[property id]&last_mod=[date]
*/
public void handleExtranetSimilarsRequest(Request request, Response response, long propId) {
   Property property = getProperty(propertyId);
   response.write("<esi:include src='/extranet-similars?property_id=" +
         propId +
         "&last_mod=" +
         property.getLastModified() +
      "'/>");
}



/*
Invoked for (intranet) requests like /similars?property_id=[property id]&last_mod=[date]
*/
public void handleSimilarsRequest(Request request, Response response, long propId) {
   if (null == request.getHeader("full_html")) {
      //This request does NOT demand that we return the actual HTML.
      // We will return a script block that will fetch the HTML via AJAX.
      response.write("<script>" +
         "dojo.addOnLoad(" +
            "function() {" +
               "dojo.xhrGet({" +
                  "url: 'http://www.redfin.com/extranet-similars?property_id=" + propId + "'," +
                  "load: function(response, ioArgs){" +
                     "dojo.byId('similar_homes').innerHTML = response;" +
                     "return response;" +
                  "}," +
                  "headers: {'full_html': 'true'}," +
                  "handleAs: 'text'" +
               "});" +
            "}" +
         ");" +
         "</script>");
      //Do NOT cache the script
      response.setCacheable(false);
   }
   else {
      //This request wants the actual HTML for similars
      response.write(getSimilarsHTML(propId));
      //The similars HTML is cacheable- that's the whole point!
      response.setCacheable(true);
   }
}


While the primary goal of most Web form designs is to get people through a form as quickly and painlessly as possible, there are situations where slowing people down is advisable. When choosing between primary and secondary actions, visual distinctions are a useful method for helping people make good choices.

Looks like LukeW has lots of other form design articles if you’re interested.