[dpdk-dev] Underlinked libs and overlinked applications - an issue?

[Christian Ehrhardt]

ok,
- little baby steps first
  I'll submit a trivial patch in reply to this fixing the underlinking in
all .so's, except librte_eal.

- discussion on the remaining open issue second.
  For the remaining issue I think that needs a special understanding of
linkers I still need to grasp.
  But since I failed to find a good explanation for the exact case we face
here I put it in a generalized form on StackOverflow so eventually more
than just us can benefit from the answer once/if we find an answer.

So to all of you - if you are not scared of the latter pages of ld/gcc man
pages :-) feel free to take a look at
http://stackoverflow.com/questions/37351699/how-to-create-both-so-files-for-two-circular-depending-libraries

For us, but not important for the question in general:
 liba = librte_eal, libb = librte_mempool
(and ring, ivshmem, but once it is solved once it is solved)

Time to stip for now, have a great Weekend
Christian



Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd

On Fri, May 20, 2016 at 2:41 PM, Christian Ehrhardt <
christian.ehrhardt at canonical.com> wrote:

> On Fri, May 20, 2016 at 1:01 PM, Panu Matilainen <pmatilai at redhat.com>
> wrote:
>
>> On 05/19/2016 09:17 PM, Thomas Monjalon wrote:
>>
>>> 2016-05-19 17:38, Christian Ehrhardt:
>>>
>>>> Hi,
>>>> I was working on the new 16.04 build system to adapt deb packaging to
>>>> it.
>>>> I remember somewhen back in the DPDK 2.2 and shared+combined library
>>>> days I
>>>> had some issues with over/underlinking - but it seems those are still
>>>> existent or came back.
>>>>
>>>
>>> I would say it has always been like that.
>>> Thanks for raising the issue.
>>>
>>> After a build in almost default config (just disabled the kernel modules)
>>>> and set RTE_MACHINE to default I find the following.
>>>>
>>>> #1
>>>> The libraries are all only linked against external things - even clearly
>>>> using internal structures:
>>>> ldd usr/lib/x86_64-linux-gnu/librte_lpm.so.2
>>>>        linux-vdso.so.1 =>  (0x00007fff7e7a5000)
>>>>        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f175d4dd000)
>>>>        /lib64/ld-linux-x86-64.so.2 (0x0000558d3afbf000)
>>>>
>>>
>>> The DT_NEEDED entries are created only for external dependencies.
>>> Probably we should create ones for internal dependencies based on the
>>> variable DEPDIRS-y.
>>>
>>
>> Yes, DT_NEEDED for internal dependencies are needed too, see eg recent
>> commits c6417ce61f83 and 8180554d82b3. That was part of Sergios' build
>> system improvement patch series last spring or so (but since then
>> abandoned), I picked up the work and been doing it in smaller pieces. Phase
>> 1 was the external dependencies, phase 2 is internal dependencies which I'm
>> working on. Unfortunately health issues have stalled my work a lot recently
>> :-/
>
>
> Since I work on 16.04 as released currently I almost missed those, but I
> at least remember seeing the first on the list.
> I had almost the same change - adopting yours since it is accepted.
> Maybe I can provide a few more baby steps on that path - or at least a few
> more pieces for discussion and review.
> I hope you get well soon Panu!
>
>
>> #2
>>>> The Application then seem to try to make up for that by realizing all
>>>> that
>>>> is missing.
>>>> But looking at the app alone it seems overlinked by that - it is not
>>>> using
>>>> all of these on its own.
>>>> ldd usr/bin/cmdline_test
>>>>        linux-vdso.so.1 =>  (0x00007ffeec9ea000)
>>>>        librte_distributor.so.1 => not found
>>>>        librte_reorder.so.1 => not found
>>>> [...]
>>>>        librte_jobstats.so.1 => not found
>>>> [...]
>>>> And for example none of the librte_jobstats.so.1 symbols are used
>>>> "directly" in there.
>>>>
>>>
>>> Yes every libraries are put for every apps in rte.app.mk.
>>> Probably that we could better use DEPDIRS-y for apps.
>>>
>>>
>> Another trick from Sergios' original patch series is to utilize
>> AS_NEEDED() in the linker script, so it ends up looking something like this
>> for shared library config (static would remain as it is now):
>>
>> INPUT ( librte_eal librte_mempool librte_ring
>>         AS_NEEDED ( librte_acl librte_timer librte_vhost <...> )
>> )
>>
>> ...and then use the linker script for linking the apps, which should
>> simplify rte.app.mk considerably. Or so the theory goes. Anyway, the
>> above requires DT_NEEDED for the internal libraries to be present first,
>> but once all these pieces are in place, dynamically linked apps will only
>> link to the libraries they actually need.
>>
>
> Interesting approach, I wasn't even working on DPDK back then - that could
> really simplify the mk there.
> I was experimenting with just dropping the "no-as-needed" in
> mk/exec-env/linuxapp/rte.vars.mk, but that seems to shoot too far -
> haven't had the time to look deeper.
> Interested if your approach could help.
>
> But the other part of it - getting the proper DT_NEEDED into the libs (I
> also consider underlinking worse than overlinking) seems to be just missing
> a bunch of -l - my last test build looked fine.
> There is the underlinking issue of librte_eal left that Sergio mentioned
> back in 6d25d90c, but it seems to be the next baby step to solve most else
> of the underlinking.
> I'll adapt it to git level and at least throw it out for discussion later
> on.
>
>