Some of the improvements we achieved include:
- A faster machine for the x64 Debug bot.
- Splitting both the x64 Debug and Release bots in build-only and test-only.
- A performance bot!.
The most interesting one is the performance bot. You can check the numbers at http://perf.webkit.org. That numbers are not valid to compare the performance between the different WebKit ports (EFL/GTK+/Mac), because each one of these performance bots run on different hardware. The idea is to track the performance between revisions, and identify the commits that have a performance impact.
The performance bot we deployed is not very powerful (i5 661), but does the job. I took extra care to ensure that external factors don’t affect the performance results of the tests on the machine. This is a summary of that measures:
- I disabled cron and every other daemon.
- Gave the performance tests processes very high priority (nice -19, ionice -c1).
- By default the performance tests are run inside Xvfb, and this is not great if you are interested in the most realistic scenario possible. So I patched WebKit to allow running the tests on the native X display.
These are some details of the bot we deployed:
- The machine is an Intel Core i5 661 with 8GB of RAM.
- As explained, we run the tests on a real X server:
$ glxinfo | grep "render.*:" direct rendering: Yes OpenGL renderer string: Mesa DRI Intel(R) Ironlake Desktop
- The OS were the tests are run is Debian Jessie.
- The DE is GNOME 3: I disabled every power-save feature of GNOME (including the screensaver). Since the window of the browser that runs the tests is positioned at the top-left of the screen, I installed the extension Activities Configurator that allows to hide the top bar of activities, and to disable the hot upper-left corner.
If you want to run the performance tests on your machine, this is as easy as:
If you want to run them on your X display, simply export the variable USE_NATIVE_XDISPLAY before:
If you don’t want to run the full suite (takes no less than 2 hours), you can specify the test names (look on the PerformanceTests directory for the names).
Tools/Scripts/run-perf-tests --platform=gtk Speedometer/Full.html