[dpdk-ci] script to determine target repo (was DPDK Lab)

[Xu, Qian Q]

yes，that's the way I want, every submitter should know their target repo

发自我的 iPhone

> 在 2017年6月26日，下午2:24，Jerin Jacob <jerin.jacob at caviumnetworks.com> 写道：
> 
> -----Original Message-----
>> Date: Fri, 23 Jun 2017 09:30:28 +0000
>> From: "Richardson, Bruce" <bruce.richardson at intel.com>
>> To: "Xu, Qian Q" <qian.q.xu at intel.com>, Thomas Monjalon
>> <thomas at monjalon.net>, "Wei, FangfangX" <fangfangx.wei at intel.com>
>> CC: "ci at dpdk.org" <ci at dpdk.org>, "O'Driscoll, Tim"
>> <tim.odriscoll at intel.com>, Eugene Voronov <eugene at mellanox.com>
>> Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
>> 
>> 
>> 
>>> -----Original Message-----
>>> From: Xu, Qian Q
>>> Sent: Friday, June 23, 2017 9:44 AM
>>> To: Thomas Monjalon <thomas at monjalon.net>; Wei, FangfangX
>>> <fangfangx.wei at intel.com>
>>> Cc: ci at dpdk.org; Richardson, Bruce <bruce.richardson at intel.com>;
>>> O'Driscoll, Tim <tim.odriscoll at intel.com>; Eugene Voronov
>>> <eugene at mellanox.com>
>>> Subject: RE: [dpdk-ci] script to determine target repo (was DPDK Lab)
>>> 
>>> Thomas/Bruce
>>> 1. For determining the repo tree to target, I don't believe that we can
>>> ever
>>>> come up with a 100% accurate rule, as the tree to which a set is to be
>>>> applied can be difficult to determine, so it may be done on the basis of
>>> on-list discussion.
>>>> A 90% accurate rule it what we may have to accept.
>>> 
>>> -- Then if we find the performance issue, then maybe it's a false alarm
>>> due to apply to the wrong repo. So, we may face many false alarms
>>> according with the time.
>>> Then people may not treat the performance issue as a problem, so I still
>>> think we need to try 100% accurate to have a more trustable result when we
>>> send out the alarm.
>> 
>> I find that rather improbable, and not worth considering. For that to per a problem multiple unlikely events have to occur:
>> 1) we mis-identify the tree on which the set is to be applied (we should be able to get to 90% accuracy here)
>> 2) the patchset must apply cleanly to the "wrong" tree (this is reasonably likely, but it's still another condition that has to be met for us to have a problem)
>> 3) the patchset has to cause a performance regression in the "wrong" tree
>> 4) but NOT cause a regression when in the right tree.
>> 
>> If we assume 90% accuracy of tree identification, optimistically that 90% of patches will apply to the wrong tree, that 5% of patches cause a performance regression (an overestimate IMHO), and that even 1/3 of those won't cause a performance regression in the right tree (a very overestimate IMHO, I would expect just about none of them to even have this), it still means that only about 1 patch in 1000 will show as a false positive performance regression.
>> 
>> 0.1 (mis-identify) * 0.9 (applies ok) * 0.05 (regression) * 0.33 (no regression) = 0.0015, or 0.15%
>> 
>> So worst case, I still don't think we have a problem for the scenario you describe.
> 
> Another option could be to fix the problem at source. Where the targeted
> script can run at the source, identify the target repo and embedded that
> information in 1st patch in series. I guess, that will provide a means
> (user can change the target repo at source if required)
> to fixup the remaining 10% accuracy.
> 
> 
> 
>>> 
>>> 
>>>> -----Original Message-----
>>>> From: Thomas Monjalon [mailto:thomas at monjalon.net]
>>>> Sent: Wednesday, June 21, 2017 4:33 PM
>>>> To: Wei, FangfangX <fangfangx.wei at intel.com>
>>>> Cc: ci at dpdk.org; Richardson, Bruce <bruce.richardson at intel.com>; Xu,
>>>> Qian Q <qian.q.xu at intel.com>; O'Driscoll, Tim
>>>> <tim.odriscoll at intel.com>; Eugene Voronov <eugene at mellanox.com>
>>>> Subject: Re: [dpdk-ci] script to determine target repo (was DPDK Lab)
>>>> 
>>>> I agree with Bruce.
>>>> 
>>>> Sorry for not having written the scripts yet.
>>>> Someone else in Mellanox should do it in July.
>>>> In the meantime, do not hesitate to share your code if it speed up
>>> things.
>>>> 
>>>> Thanks a lot
>>>> 
>>>> 
>>>> 21/06/2017 10:20, Richardson, Bruce:
>>>>> Hi Fangfang,
>>>>> 
>>>>> My comments on the questions you asked:
>>>>> 
>>>>> 1. For determining the repo tree to target, I don't believe that we
>>>>> can ever
>>>> come up with a 100% accurate rule, as the tree to which a set is to be
>>>> applied can be difficult to determine, so it may be done on the basis of
>>> on-list discussion.
>>>> A 90% accurate rule it what we may have to accept. However, since
>>>> applying a patchset to a tree should not be a time-consuming
>>>> operation, I suggest any script produce a list of possible trees in
>>>> priority order to try. If not net, then try main, etc. etc.
>>>>> 
>>>>> 2. Using the order from patch titles is correct.
>>>>> 
>>>>> /Bruce