[dpdk-dev] [PATCH] mk: fix the combined library problems by replacing it with a linker script

[Christian Ehrhardt]

Hi Ferruh,
while not tackling the "soname for combined lib" which I felt to be
the center of all this discussion.
I like that with your patch the symbols in the combined lib are no
more anonymous, but versioned according to the maps the DPDK sub
libraries are maintaining anyway.
Some more technical feedback on your post.

I wonder if we would find a similar solution for the overall soname version.
Unfortunately all naive approaches that came to my mind so far
(summing up LIBABIVER, biggest of LIBABIVER, ...) all have all too
obvious faults.
And I didn't have time for a complex one yet.
I feel that - in favor of non-perfect, but simple solution - it might
be easier to carry one global LIBABIVER for the combined lib that is
increased on each DPDK release IF (very likely) one of the ABIs was
changed in the Release.
Something like a LIBABISUBVER could be increased on any release that
did change, but not break the old ABI.
Well, likely still too naive - I need more time to really get into all
that - like to better understand all the benefits and drawbacks of the
linker script approach.

Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd


On Thu, Dec 3, 2015 at 2:31 AM, Ferruh Yigit <ferruh.yigit at intel.com> wrote:
> On Wed, Dec 02, 2015 at 06:44:19AM -0500, Neil Horman wrote:
>> On Tue, Dec 01, 2015 at 12:37:37PM +0000, Robie Basak wrote:
>> > Re-sending this unsigned since the ML rejected my signed email.
>> >
>> > -1 from Ubuntu without further discussion since it will break us. Please
>> > don't commit this patch yet.
>> >
>> > I don't understand why we must have the complexity of so many shared
>> > libraries. From a distribution packaging perspective, all I see is that
>> > this multiplies potential work by twenty times and makes it awkward to
>> > work with without special tooling (which then needs maintaining).
>> >
>> Theres nothing "complex" about the simple fact that a project builds lots of
>> libraries.  Its extreemely common. Any graphic window manager has exactly the
>> same situation, as do any number of tools that have multiple hardware backends
>> impelmented in user space (v4l, sane, iptables, to name just a few).
>>
>> > Before I go into details, it would be nice if someone could please
>> > explain why DPDK has to be "special" in needing to do this? I don't
>> Its not special, see above.  Not saying the build environment cant be improved,
>> but the fact that there are multiple libraries is pretty straightforward.
>>
>> > understand why DPDK must be different to every other userspace library
>> > out there. If DPDK has a good need to be different then that's fair
>> > enough. But I feel that if DPDK is deviating from the norm then we need
>> > to frame the discussion from the perspective of "why DPDK must be
>> > different", rather than having me trying to explain why the norm is the
>> > right way to do it.
>> >
>> > On Tue, Nov 24, 2015 at 04:31:17PM +0200, Panu Matilainen wrote:
>> > > That's how Fedora and RHEL are shipping it already and nobody has so much
>> > > as noticed anything strange, much less complained about it. 20 libraries
>> > > is but a drop in the ocean on a average distro.
>> >
>> > No, it is 20 times the work from the perspective of DPDK package
>> > maintenance. Let me explain why.
>> >
>> > In Debian and Ubuntu, we manage a library transition (an ABI bump in a
>> > library together with all dependencies moving to use the new ABI) by
>> > concurrently packaging both the old and new libraries at once. This
>> > works well with the norm for libraries. We ship one binary package per
>> > soname, with the major version as part of the package name. This allows
>> > a system to have two (or more) ABIs installed simultaneously. For a
>> > library transition, we just package the new version and then that can
>> > land and work concurrently as we then individually update every
>> > dependent (library-consuming) package.
>> So thats, a distribution choice, not an upstream problem.  And it seems like a
>> problem you should have already solved (note the examples above).  If you feel
>> like you need to package multiple ABI versions in the same library, you can,
>> just update the LIBABIVER of all the libraries, instead of the ones that truly
>> change, so that each library is guaranteed a newer so version, to make the
>> library file name unique.  Yes you have to make a small change from upstream,
>> but thats part of the work that distribution maintainers do.
>>
>>
>> > This works because of conventions around sonames, which DPDK breaks
>> > unless we treat it as twenty different libraries which changes our work
>> > from easy to painful.
>> >
>> > Usually a library transition is managed by hand by the package
>> > maintainer. It's not taxing because it's straightforward and well
>> > understood. Update and upload the new ABI source package, then find all
>> > reverse dependencies and sort them out, recursively. But if the
>> > maintainer must do it twenty times, then it becomes taxing and prone to
>> > error. And if the reverse dependency tree differs depending on the split
>> > library used by library consumers, then it gets far more complex to
>> > follow.
>> >
>> > Admittedly we could tool around this to make it easier, but that's extra
>> > work (both initially and in maintenance) and prone to error (because
>> > we'd only be doing it for DPDK).
>> >
>> You must already have a solution to this, I can't imagine you package all the
>> libraries for kde or gnome (or even pam) separately)
>>
>> > Packaging a library is usually virtually a no-op in Debian and Ubuntu
>> > nowadays. Our tooling does it all for us. But packaging DPDK is far from
>> > this currently because of all this added complexity. From my perspective
>> > this is unnecessary and makes no sense. We could do all kinds of things
>> > to work around it (that's what packaging is about) but then we'd have to
>> > maintain that specialness and I don't see why it must be awkward like
>> > this instead of just doing it the same way as every other library.
>> >
>> > > The combined library as it is simply is no longer a viable option.
>> > > Besides just being broken (witness the strange hacks people are coming
>> > > up with to work around issues in it) its ugly because it basically gives
>> > > the middle finger to all the effort going into version compatibility,
>> > > and its also big. Few projects will use every library in DPDK, but with
>> > > the combined library they're forced to lug the 800 pound gorilla along
>> > > needlessly.
>> >
>> > It's broken because it's broken upstream, and that's what we should fix.
>> > Why is it not viable? How does it give the middle finger to effort going
>> > into version compatibility?
>> Because each individual library has a version script that gets applied during
>> link to version symbols properly.  Those scripts dont get applied when building
>> the combined library.
>>
>>
>> > Doing it the right way like every other
>> > userspace library is what *gives us* version compatibility because then
>> > distributions can straightforwardly install multiple ABI versions at
>> > once.
>> Again,  Not at all uncommon.  You're packaging methodology is the issue here,
>> not the fact that there are multiple libraries.
>>
>> > Finally, I fail to see any "lug the 800 pound gorilla along" saving. We
>> > (Ubuntu and Fedora) are both shipping all the libraries in one package,
>> > whether split or combined, so they are all being lugged onto disk
>> > anyway. Whether split or combined, there is no saving there. And memory
>> > is hardly saved either because the kernel will just page in and out what
>> > is needed in both cases. So how does this proposed change give us any
>> > saving at all?
>> >
>> Not true, initalization constructors for PMD's at the very least mean that every
>> pmd will get paged in weather you want it or not using the combined library.
>> Individual libraries let you dynamically load them (via dlopen).  I think the
>> same is true of several other facets of dpdk.
>>
>> Neil
>>
> I just sent a patch that fixes versioning in combined library, can you please check it: http://dpdk.org/dev/patchwork/patch/9259/
>
> Thanks,
> ferruh
>