Webmasters Gallery

Over at the Flickr development blog, Ross Harmes , one of those lesser sung JavaScript heroes explains in detail how Flickr creates really fast client side searches and one of the implementations of these findings is the newly released find people faster feature:

The main findings of the team were that eval() is not only evil but also very slow whereas dynamic script nodes are fast but insecure. The solution was to do a custom evaluation of string data rather than using JSON:

Having set the performance bar pretty high with the last approach, we dove into custom data formats. The challenge would be to create a format that we could parse ourselves, using JavaScript’s String and RegExp methods, that would also match the speed of JSON executed natively. This would allow us to use Ajax again, but keep the data restricted to our domain.

Since we had already discovered that some methods of string manipulation didn’t perform well on large strings, we restricted ourselves to a method that we knew to be fast: split(). We used control characters to delimit each contact, and a different control character to delimit the fields within each contact. This allowed us to parse the string into contact objects with one split, then loop through that array and split again on each string.

PLAIN TEXT

JAVASCRIPT:

1. that.contacts = o.responseText.split("\c");

3. for (var n = 0, len = that.contacts.length, contactSplit; n <len; n++) {

5.         contactSplit = that.contacts[n].split("\a");

7.         that.contacts[n] = {};

8.         that.contacts[n].n = contactSplit[0];

9.         that.contacts[n].e = contactSplit[1];

10.         that.contacts[n].u = contactSplit[2];

11.         that.contacts[n].r = contactSplit[3];

12.         that.contacts[n].s = contactSplit[4];

13.         that.contacts[n].f = contactSplit[5];

14.         that.contacts[n].a = contactSplit[6];

15.         that.contacts[n].d = contactSplit[7];

16.         that.contacts[n].y = contactSplit[8];

17. }

Once this had been speeded up, all they needed to use was the YUI AutoComplete control and voilà – fast client side searches even with massive datasets.

JSONView is a new Firefox extension that gives you a nice way to view your JSON documents (JSONovich also does the trick).

Ben Hollis talks about the extension:

The extension itself is pretty simple. I wasn’t sure how to approach the problem of supporting a new content type for Firefox, so I followed the example of the wmlbrowser extension and implemented a custom nsIStreamConverter. What this means is that I created a new component that tells Firefox “I know how to translate documents of type application/json into HTML”. And that it does – parsing the JSON using the new native JSON support in Firefox 3 (for speed and security) and then constructing an HTML document that it passes along the chain. This seems to work pretty well, though there are some problems – some parts of Firefox forget the original type of the document and treat it as HTML, so “View Page Info” reports “text/html” instead of “application/json”, “Save as…” saves the generated HTML, Firebug sees the generated HTML, etc. Just recently I came across the nsIURLContentListener interface, which might offer a better way of implementing JSONView, but I’m honestly not sure – the Mozilla documentation is pretty sparse and it was hard enough to get as far as I did. I’m hoping some Mozilla gurus can give me some pointers now that it’s out in the open.