[dpdk-dev] [PATCH] mk: parallelize make config

[Bruce Richardson]

On Tue, Jan 24, 2017 at 09:42:35AM +0100, Olivier MATZ wrote:
> Hi Keith,
> 
> On Mon, 23 Jan 2017 17:50:27 +0000, "Wiles, Keith"
> <keith.wiles at intel.com> wrote:
> > > On Jan 23, 2017, at 10:18 AM, Olivier Matz <olivier.matz at 6wind.com>
> > > wrote:
> > > 
> > > Hi Ferruh,
> > > 
> > > On Sun, 22 Jan 2017 01:50:34 +0000, Ferruh Yigit
> > > <ferruh.yigit at intel.com> wrote:  
> > >> make config dependency resolving was always running serial,
> > >> parallelize it for better performance.
> > >> 
> > >> $ time make T=x86_64-native-linuxapp-gcc config
> > >> real    0m12.633s
> > >> 
> > >> $ time make -j8 T=x86_64-native-linuxapp-gcc config
> > >> real    0m1.826s
> > >> 
> > >> Signed-off-by: Ferruh Yigit <ferruh.yigit at intel.com>  
> > > 
> > > I have a patch that fix the same issue (configuration takes to
> > > long), but done differently. It is more intrusive, since it rework
> > > the way DEPDIRS are used, but it does not require to use -j.  
> > 
> > I tested the patch and the performance is very quick and seems to
> > work very nicely. I have not looked at the other patch yet are they
> > both compatible and could be combined or not?
> 
> No, I don't think they could be combined.
> 
> Ferruh's approach looks less risky, since it's a parallelization
> of the config. My approach changes the way dependencies are processed,
> and also how they are declared. I think it's faster, but more intrusive.
>
I think we may hit a case of diminishing returns. On my system, the
config time goes from 12 seconds down to less than 1 second (large
amounts of parallelism). Even with limiting parallelism to just 4
make processes, the time for config drops to 3 seconds.
I don't see huge value in trying to shave off any more time than
that, given the amount of extra complexity involved.

/Bruce