[dpdk-dev] [PATCH] mk: optimize directory dependencies

[Bruce Richardson]

On Tue, Jan 24, 2017 at 12:19:49PM +0100, Robin Jarry wrote:
> Hi Olivier,
> 
> 2017-01-23, Olivier Matz:
> > Before this patch, the management of dependencies between directories
> > had several issues:
> > 
> > - the generation of .depdirs, done at configuration is slow: it can take
> >  more than one minute on some slow targets (usually ~10s on a standard
> >  PC).
> 
> Indeed, on a Qualcomm development board where disk I/O is quite slow:
> 
>  $ git describe
>  v17.02-rc1-3-g61207d014fc9
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc
>  real    1m4.308s
> 

Wow, what is the build time in that case?

> > - for instance, it is possible to expressed a dependency like:
> 
> s/expressed/express/
> 
> > - we cannot use "make -d" for debug, because the output of make is used
> > for the generation of .depdirs.
> 
> That is really annoying when debugging makefiles.
> 
> After applying this patch:
> 
>  $ git am mk-optimize-directory-dependencies.patch
>  Applying: mk: optimize directory dependencies
>  $ rm -rf build/
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc
>  real    0m0.111s
> 
> Almost 600 times faster than before!
> 
> I prefer this solution to the one proposed by Ferruh (which is interesting
> but requires to run parallel make). Here is a test with the other patch:
> 
>  $ git am mk-parallelize-make-config.patch
>  Applying: mk: parallelize make config
>  $ rm -rf build/
>  $ grep -c processor /proc/cpuinfo
>  24
> 
>  $ time make config T=arm64-armv8a-linuxapp-gcc -j24
>  real    0m11.253s
> 
> Here only 6 times faster than before, even when using 24 parallel processes.
> 
> Tested-by: Robin Jarry <robin.jarry at 6wind.com>

Hi Robin,

what are the differences in the patches like when doing a build rather
than just a config? If the build is minutes long because of slow IO,
is the extra 10 seconds really going to make that much of a difference?

Regards,
/Bruce