Edit: Due to the level of interest, we’ve released the source code to the work described here: https://github.com/scalyr/angular.

Here at Scalyr, we recently embarked on a full rewrite of our web client. Our application is a broad-spectrum monitoring and log analysis tool. Our home-grown log database executes most queries in tens of milliseconds, but each interaction required a page load, taking several seconds for the user.

A single-page application architecture promised to unlock the backend’s blazing performance, so we began searching for an appropriate framework, and identified AngularJS as a promising candidate. Following the “fail fast” principle, we began with our toughest challenge, the log view.

This is a real acid test for an application framework. The user can click on any word to search for related log messages, so there may be thousands of clickable elements on the page; yet we want instantaneous response for paging through the log. We were already prefetching the next page of log data, so the user interface update was the bottleneck. A straightforward AngularJS implementation of the log view took 1.2 seconds to advance to the next page, but with some careful optimizations we were able to reduce that to 35 milliseconds. These optimizations proved to be useful in other parts of the application, and fit in well with the AngularJS philosophy, though we had to break few rules to implement them. In this article, we’ll discuss the techniques we used.

An AngularJS log viewer

At heart, the Log View is simply a list of log messages. Each word is clickable, and so must be placed in its own DOM element. A simple implementation in AngularJS might look like this:

One page can easily have several thousand tokens. In our early tests, we found that advancing to the next log page could take several agonizing seconds of JavaScript execution. Worse, unrelated actions (such as clicking on a navigation dropdown) now had noticeable lag. The conventional wisdom for AngularJS says that you should keep the number of data-bound elements below 200. With an element per word, we were far above that level.

Analysis

Using Chrome’s Javascript profiler, we quickly identified two sources of lag. First, each update spent a lot of time creating and destroying DOM elements. If the new view has a different number of lines, or any line has a different number of words, Angular’s ng-repeat directive will create or destroy DOM elements accordingly. This turned out to be quite expensive.

Second, each word had its own change watcher, which AngularJS would invoke on every mouse click. This was causing the lag on unrelated actions like the navigation dropdown.

Optimization #1: Cache DOM elements

We created a variant of the ng-repeat directive. In our version, when the number of data elements is reduced, the excess DOM elements are hidden but not destroyed. If the number of elements later increases, we re-use these cached elements before creating new ones.

Optimization #2: Aggregate watchers

All that time spent invoking change watchers was mostly wasted. In our application, the data associated with a particular word can never change unless the overall array of log messages changes. To address this, we created a directive that “hides” the change watchers of its children, allowing them to be invoked only when the value of a specified parent expression changes. With this change, we avoided invoking thousands of per-word change watchers on every mouse click or other minor event.  (To accomplish this, we had to slightly break the AngularJS abstraction layer. We’ll say a bit more about this in the conclusion.)

Optimization #3: Defer element creation

As noted, we create a separate DOM element for each word in the log. We could get the same visual appearance with a single DOM element per line; the extra elements are needed only for mouse interactivity. Therefore, we decided to defer the creation of per-word elements for a particular line until the mouse moves over that line.

To implement this, we create two versions of each line. One is a simple text element, showing the complete log message. The other is a placeholder which will eventually be populated with an element per word. The placeholder is initially hidden. When the mouse moves over that line, the placeholder is shown and the simple version is hidden. Showing the placeholder causes it to be populated with word elements, as described next.

Optimization #4: Bypass watchers for hidden elements

We created one more directive, which prevents watchers from being executed for an element (or its children) when the element is hidden. This supports Optimization #1, eliminating any overhead for extra DOM nodes which have been hidden because we currently have more DOM nodes than data elements. It also supports Optimization #3, making it easy to defer the creation of per-word nodes until the tokenized version of the line is shown.

Here is what the code looks like with all these optimizations applied. Our custom directives are shown in bold.

sly-repeat is our variant of ng-repeat, which hides extra DOM elements rather than destroying them. sly-evaluate-only-when prevents inner change watchers from executing unless the “logLines” variable changes, indicating that the user has advanced to a new section of the log. And sly-prevent-evaluation-when-hidden prevents the inner repeat clause from executing until the mouse moves over this line and the div is displayed.

This shows the power of AngularJS for encapsulation and separation of concerns. We’ve applied some fairly sophisticated optimizations without much impact on the structure of the template. (This isn’t the exact code we’re using in production, but it captures all of the important elements.)

Results

To evaluate performance, we added code to measure the time from a mouse click, until Angular’s $digest cycle finishes (meaning that we are finished updating the DOM). The elapsed time is displayed in a widget on the side of the page. We measured performance of the “Next Page” button while viewing a Tomcat access log, using Chrome on a recent MacBook Pro. Here are the results (each number is the average of 10 trials):

These figures do not include the time the browser spends in DOM layout and repaint (after JavaScript execution has finished), which is around 30 milliseconds in each implementation. Even so, the difference is dramatic; Next Page time dropped from a “stately” 1.2 seconds, to an imperceptible 35 ms (65 ms with rendering).

The “data fetched from server” figures include time for an AJAX call to our backend to fetch the log data. This is unusual for the Next Page button, because we prefetch the next page of logs, but may be applicable for other UI interactions. But even here, the optimized version updates almost instantly.

Conclusion

This code has been in production for two months, and we’re very happy with the results. You can see it in action at the Scalyr Logs demo site. After entering the demo, click the “Log View” link, and play with the Next / Prev buttons. It’s so fast, you’ll find it hard to believe you’re seeing live data from a real server.

Implementing these optimizations in a clean manner was a fair amount of work. It would have been simpler to create a single custom directive that directly generated all of the HTML for the log view, bypassing ng-repeat. However, this would have been against the spirit of AngularJS, bearing a cost in code maintainability, testability, and other considerations. Since the log view was our test project for AngularJS, we wanted to verify that a clean solution was possible. Also, the new directives we created have already been used in other parts of our application.

We did our best to follow the Angular philosophy, but we did have to bend the AngularJS abstraction layer to implement some of these optimizations. We overrode the Scope’s $watch method to intercept watcher registration, and then had to do some careful manipulation of Scope’s instance variables to control which watchers are evaluated during a $digest.

Next time

This article covered a set of techniques we used to optimize JavaScript execution time in our AngularJS port. We’re big believers in pushing performance to the limit, and these are just some of the tricks we’ve used. In upcoming articles, we’ll describe techniques to reduce network requests, network latency, and server execution time. We may also discuss our general experience with AngularJS and the approach we took to structuring our application code — if you’re interested in this, let us know in the comments. For more discussion on AngularJS performance, you can also read this article by Gleb Bahmutov.

Obligatory plug

At Scalyr, we’re all about improving the DevOps experience through better technology. If you’ve read this far, you should probably read more about the Scalyr log management tool.