Mark @ Mozilla

Performance team

The Final Countdown

Release day is rapidly approaching. We’re targeting October 1st for the release of the first version of the telemetry reboot. Only two weeks left – exciting times!

There’s now a tracking bug that should list everything that needs to be done before things go live: Bug 911300.

The code has changed a fair bit since my last update, the most noticeable modification being that the HTTP server is now using node.js instead of a python+flask server.

Here is the basic architecture diagram for the system: Telemetry Data Flow diagram

Data flows into an incoming bucket, which can then be processed separately by one or more processing nodes, each of which publishes finalized data to a published bucket.

The MapReduce code then reads from the published location for data analysis.

Other changes include updating the “process incoming data” code to take advantage of multiple processors, though it appears that python performance is still a major bottleneck. Fortunately Mike Trinkala is working on porting the conversion code to C++.

Keep an eye on the bug above for up-to-the-minute progress information, or as always, feel free to drop by #telemetry on irc.mozilla.org.

Comments