[dpdk-stable] [dpdklab] RE: [dpdk-dev] [PATCH v4] mbuf: fix reset on mbuf free

[Ferruh Yigit]

On 1/21/2021 4:35 PM, Lincoln Lavoie wrote:
> Hi All,
> 
> Trying to follow the specific conversation.  It is correct, the lab does not 
> list the specific throughput values achieved by the hardware, as that data can 
> be sensitive to the hardware vendors, etc. The purpose of the lab is to check 
> for degradations caused by patches, so the difference is really the important 
> factor.  The comparison is against a prior run on the same hardware, via the 
> DPDK main branch, so any delta should be caused by the specific patch changes 
> (excluding statistical "wiggle").
> 
> If the group would prefer, we could calculate additional references if desired 
> (i.e. difference from the last official release, or a monthly run of the 
> current, etc.).  We just need the community to define their needs, and we can 
> add this to the development queue.
> 

Hi Brandon,

Can you also put above to the backlog, to display the performance difference to 
the a fixed point, like a previous release or a previous LTS?

Thanks,
ferruh

> Cheers,
> Lincoln
> 
> 
> On Thu, Jan 21, 2021 at 4:29 AM Morten Brørup <mb at smartsharesystems.com 
> <mailto:mb at smartsharesystems.com>> wrote:
> 
>      > From: dev [mailto:dev-bounces at dpdk.org <mailto:dev-bounces at dpdk.org>] On
>     Behalf Of Ferruh Yigit
>      > Sent: Thursday, January 21, 2021 10:19 AM
>      >
>      > On 1/15/2021 6:39 PM, Ali Alnubani wrote:
>      > > Hi,
>      > > Adding Ferruh and Zhaoyan,
>      > >
>      > >> Ali,
>      > >>
>      > >> You reported some performance regression, did you confirm it?
>      > >> If I get no reply by monday, I'll proceed with this patch.
>      > >
>      > > Sure I'll confirm by Monday.
>      > >
>      > > Doesn't the regression also reproduce on the Lab's Intel servers?
>      > > Even though the check iol-intel-Performance isn't failing, I can see
>      > that the throughput differences from expected for this patch are less
>      > than those of another patch that was tested only 20 minutes earlier.
>      > Both patches were applied to the same tree:
>      > >
>      > > https://mails.dpdk.org/archives/test-report/2021-January/173927.html
>     <https://mails.dpdk.org/archives/test-report/2021-January/173927.html>
>      > >> | 64         | 512     | 1.571                               |
>      > >
>      > > https://mails.dpdk.org/archives/test-report/2021-January/173919.html
>     <https://mails.dpdk.org/archives/test-report/2021-January/173919.html>
>      > >> | 64         | 512     | 2.698                               |
>      > >
>      > > Assuming that pw86457 doesn't have an effect on this test, it looks
>      > to me that this patch caused a regression in Intel hardware as well.
>      > >
>      > > Can someone update the baseline's expected values for the Intel NICs
>      > and rerun the test on this patch?
>      > >
>      >
>      > Zhaoyan said that the baseline is calculated dynamically,
>      > what I understand is baseline set based on previous days performance
>      > result, so
>      > it shouldn't require updating.
> 
>     That sounds smart!
> 
>     Perhaps another reference baseline could be added, for informational
>     purposes only:
>     Deviation from the performance of the last official release.
> 
>      >
>      > But cc'ed the lab for more details.
> 
> 
> 
> -- 
> *Lincoln Lavoie*
> Senior Engineer, Broadband Technologies
> 21 Madbury Rd., Ste. 100, Durham, NH 03824
> lylavoie at iol.unh.edu <mailto:lylavoie at iol.unh.edu>
> https://www.iol.unh.edu <https://www.iol.unh.edu>
> +1-603-674-2755 (m)
> <https://www.iol.unh.edu>