By Jeremy Plsek, Lincoln LaVoie and Patrick MacArthur
The DPDK Community Lab is an open, independent testing resource for the DPDK project. Its purpose is to perform automated testing on incoming patch submissions, to ensure the performance and quality of DPDK is maintained. Participation in the lab is open to all DPDK project participants.
For some time now, the DPDK Community Lab has been gathering performance deltas using the single-core packet I/O layer 2 throughput test from DTS for each patch series submitted to DPDK compared to the master branch. We are pleased to announce that the Lab has recently been allowed to make these results public. These results are also now published to Patchwork as they are automatically generated. These results currently contain Mellanox and Intel devices, and the lab is able to support hardware from any DPDK participants wishing to support these testing efforts.
To view these results, you can go to DPDK Community Lab Dashboard via the following link: https://lab.dpdk.org. The dashboard lists an overview of all active patch series and their results. Detailed results can be viewed by clicking on the patch series. If a patch fails to merge into master, a build log will show to help identify any issues. If a patch cleanly merges into master, performance delta results will show for each participating member.
The Lab is hosted by the University of New Hampshire InterOperability Laboratory, as a neutral, third party location. This provides a secure environment for hosting equipment and generating unbiased results for all participating vendors. Lab participants, i.e. companies hosting equipment in the testing, can securely access their equipment through a VPN, allowing for maintenance and performance tuning, as the DPDK project progresses.
The Lab works by polling the Patchwork API. When new patches are submitted, the CI server merges them with the master branch and generates a tarball. Each participating system unpacks and installs the DPDK tarball and then runs the performance testing against this DPDK build. When all systems have finished testing, the CI gathers the results into our internal database to be shown on the Dashboard, and sends final reports to Patchwork to show up on the submitted patch. This allows patch submitters to utilize Patchwork to view their individual results, while also allowing anyone to quickly see an overview of results on the Dashboard. The system provides maintainers with positive confirmation of the stability and performance of the overall project.
In the future, we plan to open the Lab to more testing scenarios, such as performance testing of other features, beyond single-core packet I/O layer 2 throughput, and possibly running Unit Tests for DPDK. Additional features will be added to the Dashboard, such as showing graphs of the performance changes of master over time.
If your company would like to be involved, email the Continuous Integration group at ci@dpdk.org and dpdklab@iol.unh.edu.