[PATCH v4] devtools: parallelize ABI check

[David Marchand]

On Thu, Jan 12, 2023 at 3:15 PM Ferruh Yigit <ferruh.yigit at amd.com> wrote:
> >>         abidiff $ABIDIFF_OPTIONS $dump $dump2 || {
> >>                 abiret=$?
> >> -               echo "Error: ABI issue reported for 'abidiff $ABIDIFF_OPTIONS $dump $dump2'" >&2
> >> -               error=1
> >> -               echo
> >> +               echo "Error: ABI issue reported for abidiff $ABIDIFF_OPTIONS $dump $dump2" >&2
> >>                 if [ $(($abiret & 3)) -ne 0 ]; then
> >>                         echo "ABIDIFF_ERROR|ABIDIFF_USAGE_ERROR, this could be a script or environment issue." >&2
> >>                 fi
> >> @@ -57,8 +55,13 @@ for dump in $(find $refdir -name "*.dump"); do
> >>                 if [ $(($abiret & 8)) -ne 0 ]; then
> >>                         echo "ABIDIFF_ABI_INCOMPATIBLE_CHANGE, this change breaks the ABI." >&2
> >>                 fi
> >> -               echo
> >> +               return 1
> >>         }
> >> -done
> >> +}'
> >> +
> >> +error=
> >> +find $refdir -name "*.dump" |
> >> +xargs -n1 -P0 sh -c 'eval "$diff_func"; run_diff $0' ||
> >> +error=1
> >>
> >>  [ -z "$error" ] || [ -n "$warnonly" ]
> >
> > For the record, on my system, calling this script is ~5 times faster:
> > - before
> > real    0m5,447s
> > user    0m4,497s
> > sys    0m0,937s
> >
> > - after
> > real    0m1,202s
> > user    0m10,784s
> > sys    0m2,027s
> >
> >
> >> diff --git a/devtools/gen-abi.sh b/devtools/gen-abi.sh
> >> index f15a3b9aaf..61f7510ea1 100755
> >> --- a/devtools/gen-abi.sh
> >> +++ b/devtools/gen-abi.sh
> >> @@ -22,5 +22,6 @@ for f in $(find $installdir -name "*.so.*"); do
> >>         fi
> >>
> >>         libname=$(basename $f)
> >> -       abidw --out-file $dumpdir/${libname%.so*}.dump $f
> >> -done
> >> +       echo $dumpdir/${libname%.so*}.dump $f
> >> +done |
> >> +xargs -n2 -P0 abidw --out-file
> >> --
> >> 2.39.0
> >>
> >
> > - before
> > real    0m8,237s
> > user    0m7,704s
> > sys    0m0,504s
> >
> > - after
> > real    0m2,517s
> > user    0m14,145s
> > sys    0m0,766s
> >
> >
> > Ferruh, I am seeing quite different numbers for running those scripts
> > (clearly not of the minute order).
> > I switched to testing/building in tmpfs some time ago.
> > It requires a good amount of memory (I empirically allocated 40G), but
> > maybe worth a try for you?
> >
>
> I run 'test-meson-builds.sh' script directly and yes I am getting
> different numbers although there is still improvement, not in scale with
> what you are getting, with v4 I have following:
>
> - before
> real    10m3.248s
> user    39m8.664s
> sys     14m52.870s
>
> - after
> real    7m49.086s
> user    39m59.507s
> sys     15m0.598s

Well, yes, I did not realise which apples you were looking at :-).
The change looks good in any case.


-- 
David Marchand