[dpdk-dev] [PATCH v3 7/8] mk: sort object files when building deps lists

[Luca Boccassi]

On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote:
> 27/06/2017 12:43, Luca Boccassi:
> > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote:
> > > 23/06/2017 20:41, lboccass at brocade.com:
> > > > From: Luca Boccassi <luca.boccassi at gmail.com>
> > > > 
> > > > In order to achieve reproducible builds, always use the same
> > > > order when listing object files to build dependencies lists.
> > > > 
> > > > Signed-off-by: Luca Boccassi <luca.boccassi at gmail.com>
> > > > ---
> > > >  mk/rte.app.mk     | 4 ++--
> > > >  mk/rte.hostapp.mk | 4 ++--
> > > >  mk/rte.shared.mk  | 4 ++--
> > > >  3 files changed, 6 insertions(+), 6 deletions(-)
> > > > 
> > > > --- a/mk/rte.app.mk
> > > > +++ b/mk/rte.app.mk
> > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)-
> > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB
> > > >  
> > > >  # list of found libraries files (useful for deps). If not
> > > > found,
> > > > the
> > > >  # library is silently ignored and dep won't be checked
> > > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\
> > > > -	$(addprefix $(dir)/,$(LDLIBS_NAMES))))
> > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach
> > > > dir,$(LDLIBS_PATH),\
> > > > +	$(addprefix $(dir)/,$(LDLIBS_NAMES)))))
> > > 
> > > You cannot sort libraries.
> > > Check - for instance - this comment above in this file:
> > > 	# Eliminate duplicates without sorting, only keep the last
> > > occurrence
> > > 	filter-libs = \
> > 
> > Not sure I follow - what's the reason for avoiding to sort the list
> > of
> > libs to link against?
> 
> Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES.
> 
> > > Why sorting them?
> > > What is random in libraries list?
> > 
> > The issue is that the output of wildcard is not fully
> > deterministic. It
> > can depend on the filesystem, and even on the locale settings [1].
> > Before GNU Make 3.82 (2009) it used to automatically sort the
> > output,
> > but that was removed (to make it faster... sigh). [2]
> 
> It is not a true wildcard here. It is just filtering files which
> do not exist.
> I think you do not need this patch for deterministic build.

But then those lists are passed down in the .SECONDEXPANSION rule
right?

I am trying to find out an alternative solution. The problem to solve
is that the build system picks the public headers path (which is
embedded in the object files as notation and in the debug symbol) from
a seemingly random location between the make install path and the
source path (build/include/rte_foo.h vs lib/librte_foo/rte_foo.h) and
this makes the build not reproducible.

Nonetheless, while I work more on the last 4 patches, could you please
have a look and eventually take patch 3 and 4? They are needed to
respectively have a deterministic list of files in the libdpdk.so
linker script and a list of source files in one of the example
documentation files.

Thanks!

-- 
Kind regards,
Luca Boccassi