[dpdklab] RE: rte_service unit test failing randomly

[Morten Brørup]

From: Lincoln Lavoie [mailto:lylavoie at iol.unh.edu] 
Sent: Thursday, 6 October 2022 13.07
> On Thu, Oct 6, 2022 at 5:49 AM Morten Brørup <mb at smartsharesystems.com> wrote:
> > From: Mattias Rönnblom [mailto:mattias.ronnblom at ericsson.com]
> > Sent: Thursday, 6 October 2022 10.59
> > 
> > On 2022-10-06 10:18, Morten Brørup wrote:
> > >> From: Mattias Rönnblom [mailto:mattias.ronnblom at ericsson.com]
> > >> Sent: Thursday, 6 October 2022 09.51
> > >>
> > >> On 2022-10-06 08:53, Morten Brørup wrote:
> > >
> > > [...]
> > >
> > >>> I have been wondering how accurate the tests really are. Where can
> > I
> > >> see what is being done to ensure that the EAL worker threads are
> > fully
> > >> isolated, and never interrupted by the O/S scheduler or similar?

[...]

> Back to the topic of unit testing, I think we need to consider their purpose and where we expect them to run.  Unit tests are run in automated environments, across multiple CI systems, i.e. UNH-IOL Community Lab, GitHub, etc.  Those environments are typically virtualized and I don't think the unit tests should require turning down to the level of CPU clock ticks.  Those tests are likely better suited to dedicated performance environments, where the complete host is tightly controlled, for the purpose of repeatable and deterministic results on things like packet throughput, etc.

Excellent point, Lincoln. Verifying the performance of the surrounding runtime environment - i.e. the host running the tests - is not unit test material, it belongs with the performance tests.

So, how do I add a test case to the performance test suite? Is there a guide, a reference example, or any other documentation I can look at for inspiration?

NB: This is a good example of yesterday's techboard meeting discussion... I'm a DTS newbie wanting to contribute with a new test case, how do I get started?

If no such documentation exists, could someone please point at a couple of files representing what needs to be added?