[dpdk-dev] [dpdk-ci] [RFC] Proposal for allowing rerun of tests
David Marchand
david.marchand at redhat.com
Tue Apr 13 16:59:00 CEST 2021
On Tue, Apr 13, 2021 at 4:47 PM Thomas Monjalon <thomas at monjalon.net> wrote:
>
> 13/04/2021 15:50, Aaron Conole:
> > During the various CI pipelines, sometimes a test setup or lab will
> > have an internal failure unrelated to the specific patch. Perhaps
> > 'master' branch (or the associated -next branch) is broken and we cannot
> > get a successful run anyway. Perhaps a network outage occurs during
> > infrastructure setup. Perhaps some other transient error clobbers the
> > setup. In all of these cases the report to the mailing flags the patch
> > as 'FAIL'.
> >
> > It would be very helpful if maintainers had the ability to tell various
> > CI infrastructures to restart / rerun patch tests. For now, this has to
> > be done by the individual managers of those labs. Some labs, it isn't
> > possible. Others, it's possible but is a very time-consuming process to
> > restart a test case. In all cases, a maintainer needs to spend time
> > communicating with a lab manager. This could be made a bit nicer.
>
Yes, this is something that is often discussed with other maintainers.
>
> > One proposal we (Michael and I) have toyed with for our lab is having
> > the infrastructure monitor patchwork comments for a restart flag, and
> > kick off based on that information. Patchwork tracks all of the
> > comments for each patch / series so we could look at the series that
> > are still in a state for 'merging' (new, assigned, etc) and check the
> > patch .comments API for new comments. Getting the data from PW should
> > be pretty simple - but I think that knowing whether to kick off the
> > test might be more difficult. We have concerns about which messages we
> > should accept (for example, can anyone ask for a series to be rerun, and
> > we'll need to track which rerun messages we've accepted). The
> > convention needs to be something we all can work with (ie: /Re-check:
> > [checkname] or something as a single line in the email).
> >
> > This is just a start to identify and explain the concern. Maybe there
> > are other issues we've not considered, or maybe folks think this is a
> > terrible idea not worth spending any time developing. I think there's
> > enough use for it that I am raising it here, and we can discuss it.
>
> First question: WHO should be allowed to ask for a re-run?
> - everybody
> - patchwork delegate
Patchwork delegate requires to maintain a map between pw logins and an
actual mail address (if we go with email for the second point).
> - a list of maintainers
I'd vote on any maintainer from MAINTAINERS, _but_ it must be from the
files in the repo, not in the series being tested.
So maybe the easier is to have an explicit list... ?
- author
Just listing this option for discussion, but this is dangerous, as any
user could then call reruns.
>
> Second question: HOW requesting a re-run?
> - comment in email with formatted message
> - patchwork button
> - postal letter
While the postal letter has its charm, an email on the ml is better
than pw for me.
It leaves a trace on who asked and when.
And I am not sure how you could trigger a CI rerun with patchwork anyway :-).
>
> Third question: WHERE hosting this mechanism?
> - only one answer: in dpdk-ci.git consumed by labs
--
David Marchand
More information about the dev
mailing list