Dojo Phased Javascript Loading

Here at Redfin, we undertook a rather large project to redesign one of our most visited pages: the details of a home. This was no easy task but one of the main choices we made early on in the redesign was to fully embrace javascript for handling all of our view code.

While this approach is very appealing from a developers point of view of harmonizing the programming languages for building our pages, we were concerned about the size of the javascript required to be downloaded, parsed and ready to go before even showing anything to our users. In fact, we ran into this issue before with our map page. It had ballooned so much that it required over a MB of JS to be downloaded before anything was rendered on the page.

So, with the new design and the inevitable addition of more and more widgets, we took some time and figured out a strategy for handling a page that requires a lot of javascript. Not only that, we wanted to prioritize the “above the fold” content to render as fast as possible.

Dojo Parsing

There are a couple approaches to creating widgets with in dojo. One is to use the parseOnLoad flag of dojo to signal to it that it should parse the DOM once dojo itself and all the dojo.require’s have finished. Another is to hook directly into the dojo.addOnLoad function callback and start creating your own widgets. In both of these cases, in a production environment, we will have to require a main profile rollup of all our javascript code.

Institutionalizing Javascript Phases

However, since we wanted to split up the way we did our rollups, dojo didn’t necessary provide us an easy mechanism when certain JS definitions do not exist yet. At first we looked at trapping the errors of missing widgets in parseOnLoad and just re-invoking dojo.parse.parser() once we got the requires but while this somewhat accomplished what we wanted, it wasn’t quite an elegant solution to our problem. It masked other errors that might have happened with the widget creation.

So we looked deeper at the parser and started pulling out parts it so we could control when and where we wanted to invoke the parser.

A different Dojo type parser

We decided we would divide up our widgets so that the top of the fold widgets would be in one grouping, the middle content in another grouping and any leftover widgets in the last grouping. Internally we call these groupings “phases”. It’s arbitrary in how many phases we have and what widgets we assign to what phase, but we divided them into three main phases so we could minimize the amount of network requests while still having the top of the fold and middle fold content render as early as possible.

The order in which this parser works is pretty simple. First, it will start executing once dojo.addOnLoad invokes our callback. This essentially means for us that it’s pulled down our main dojo.js rollup and the first “phase” rollup. This enables us to start bootstrapping the page by sending out data requests (see our data loader post later!) and then for the leftover javascript rollups. The key here is that we don’t eval the javascript yet, just store it until we’re ready. It may be the case that the javascript is already cached but we still want to render the page in a certain order so that the users will see the top of the fold content first and as early as possible.

So, after we’ve sent out our network requests, we can start the first widget creation phase. The majority of our widgets extend from a widget that is aware that we’re loading our javascript in phases. This widget overrides the buildRendering phase. We disable _Templated’s scanning for children and just inform our parser that the widget it just created has children that need to be created. The parser does this for each widget it creates, collecting each new dojoType within the templates of the widgets. When it collects these types, it first checks to see if it can create it (does the class exist) and if not, saves it off to be reparsed in the subsequent phase loading.

Once the parser finishes up its first phase scan and widget creation, it will invoke the startup() method on all the widgets. This is pretty much inline with the traditional life-cycle definition of startup(). After all the widgets have been informed, the parser will eval the next phase of javascript and loop through all the saved off dom nodes that it did not find the dojo class for yet.

Upon reaching the last phase, if there are still classes left over, we put in some debugging statements that help us identify any dojoTypes that were never created. We also adjusted the creation of the widgets to not blow up the whole page meaning that whenever there is an error while creating a widget, it will be removed from the list and move on.

All this gives us the ability to render each of our phases, attach them to the DOM and present to the user our content as soon as possible.

_delayedAdopt

The last little detail with this setup is programmatic creation. We had to get a little clever so that we could create something once all the javascript rollups have been downloaded. Inspired by Higgin’s article for _Adopt (something that we also use for programmatic creation), we created a second function that acts just like adopt() but uses a string for the class name.

delayedAdopt: function(/* string */cls, /* object? */props, /* dom node */node, /* function */callback) {}

The callback is automatically bound to the context it was created in just like the normal adopt() but will not execute until that the require for that widget has been pulled down. This lets us make the creation of widgets that are not necessary in the top of the fold content be created in a later phase.

Phased loading

With all this together, this gives us the ability to split up our javascript into chunks and force an order of creating the page. It reduces the initial page size down so that the user is interacting and seeing something almost immediately. Most people eat with their eyes and dislike pages that seem to be loading slowly. Hopefully this makes it seem like the page loads faster for users.

Discussion