Lately I have been working on the WebKitGTK+ build bots that Igalia maintains.
Some of the improvements we achieved include:
- A faster machine for the x64 Debug bot.
- Splitting both the x64 Debug and Release bots in build-only and test-only.
- A performance bot!.
The most interesting one is the performance bot. You can check the numbers at http://perf.webkit.org. That numbers are not valid to compare the performance between the different WebKit ports (EFL/GTK+/Mac), because each one of these performance bots run on different hardware. The idea is to track the performance between revisions, and identify the commits that have a performance impact.
The performance bot we deployed is not very powerful (i5 661), but does the job. I took extra care to ensure that external factors don’t affect the performance results of the tests on the machine. This is a summary of that measures:
- I disabled cron and every other daemon.
- Gave the performance tests processes very high priority (nice -19, ionice -c1).
- Disabled Hyper-Threading: While the tests are run in sequential order, the javascript engine (JSC) or other parts of WebKit can be parallelized. HT is great if you want to squeeze the maximum performance of your machine, but it can become a problem when you care about consistent results related to performance (or you care about real time).
- By default the performance tests are run inside Xvfb, and this is not great if you are interested in the most realistic scenario possible. So I patched WebKit to allow running the tests on the native X display.
These are some details of the bot we deployed:
- The machine is an Intel Core i5 661 with 8GB of RAM.
- As explained, we run the tests on a real X server:
$ glxinfo | grep "render.*:" direct rendering: Yes OpenGL renderer string: Mesa DRI Intel(R) Ironlake Desktop
- The OS were the tests are run is Debian Jessie.
- The DE is GNOME 3: I disabled every power-save feature of GNOME (including the screensaver). Since the window of the browser that runs the tests is positioned at the top-left of the screen, I installed the extension Activities Configurator that allows to hide the top bar of activities, and to disable the hot upper-left corner.
If you want to run the performance tests on your machine, this is as easy as:
Tools/Scripts/run-perf-tests --platform=gtk
If you want to run them on your X display, simply export the variable USE_NATIVE_XDISPLAY before:
export USE_NATIVE_XDISPLAY=1
If you don’t want to run the full suite (takes no less than 2 hours), you can specify the test names (look on the PerformanceTests directory for the names).
For example to run the test DoYouEvenBench Speedometer (DoYouEvenBench was renamed to Speedometer):
Tools/Scripts/run-perf-tests --platform=gtk Speedometer/Full.html
Check the WebKit wiki for further information and keep and eye on http://perf.webkit.org: If you notice any performance regression, please let us know that by opening a bug.
