[dpdk-dev] Further fun with ABI tracking

[Jan Blunck]

On Tue, Feb 14, 2017 at 11:52 AM, Christian Ehrhardt
<christian.ehrhardt at canonical.com> wrote:
> Hi,
> when moving to DPDK 16.11 Debian/Ubuntu packaging of DPDK has hit a new
> twist on the (it seems reoccurring) topic of DPDK ABI tracking.
>
> I have found, ... well I don't want to call it solution ..., let's say a
> crutch to get around it for the moment. But I wanted to use the example I
> had to share a few thoughts on it and to kick off a wider discussion.
>
>
> *## In library cross-dependencies plus partial ABI bumps ##*
>
> Since the day moving away from the combined shared library we had several
> improvements on tracking the ABI versions. These days [1] we have LIBABIVER
> per library and it gets bumped to reflect it is breaking with former
> versions e.g. removing symbols.
>
> Now in the 16.11 release the ABIs for cryptodev, eal and ethdev got bumped
> by [2] and [3].
>
> OTOH please remember that in general two versions of a shared library in
> the usual sense are meant to be able to stay alongside on a system without
> hurting each other. I picked a random one on my system.
> Package              Library
> libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160
> libisc-export160: /lib/x86_64-linux-gnu/libisc-export.so.160.0.0
> libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95
> libisc-export95: /lib/x86_64-linux-gnu/libisc-export.so.95.5.0
> Some link against the new, some against the old library - all fine.
> Usually most programs can just be rebuilt against the new library and after
> some time the old one can be dropped. That mechanism gives downstream
> distributions a way to handle transitions and consumers of libraries which
> might not all be ready for the same version every time.
> And since the per lib versioning with LIBABIVER and and the version maps we
> are good - in fact we qualify for all common cases on [4].
>
> Now in DPDK of those libraries that got an ABI bump eal and ethdev are part
> of those which most of us consider "core libraries" and most other libs and
> pmds link to them.
> And here DPDK continues to be special, due to that inter-dependency with
> old and new libraries installed on the same system the following happens on
> openvswitch built for an older version of dpdk:
> ovs-vswitchd-dpdk
>     librte_eal.so.2 => /usr/lib/x86_64-linux-gnu/librte_eal.so.2
>     librte_pdump.so.1 => /usr/lib/x86_64-linux-gnu/librte_pdump.so.1
>         librte_eal.so.3 => /usr/lib/x86_64-linux-gnu/librte_eal.so.3
>
> You can see that Openvswitch itself depends on the "old" librte_eal.so.2.
> But because  librte_pdump.so.1 did not get an ABI bump it got upgraded to
> the newer version from DPDK 16.11.
> But since the "new" pdump got built with the new DPDK 16.11 it depends on
> the "new" librte_eal.so.3.
> And having both in the same executable space at the same time causes
> segfaults and pain.
>
> As I said for now I have passed the issue with a crutch that I'm not proud
> of and I'd like to avoid in the future. For that I'm reaching out to you
> with several suggestions to discuss.
>
>
> *## Thoughts ##*
> None of these seems like a perfect solution to me yet, but clearly good to
> start discussions on them.
>
> Options that were in discussion so far and that we might adopt next cycle
> (some of these are upstream changes, some downstream, some require both to
> change - but any of them should have an ack upstream so that we are
> agreeing how to proceed with those cases).
>
> 1. Downstreams to insert Major version into soname
> Distributions could insert the DPDK major version (like 16.11) into the
> soname and package names. A common example of this is libboost [5].
> That would perfectly allow 16.07.<LIBABIVER> to coexist with
> 16.11.<LIBABIVER> even if for a given library LIBABIVER did not change.
> Yet it would mean that anything depending on the old library will have to
> be recompiled to pick up the new code, even if it depends on an ABI that is
> still present in the new release.
> Also - not a technical reason - but it is clearly more work to force update
> all dependencies and clean out old packages for every release.

Actually this isn't exactly what I proposed during the summit. Just
keep it simple and fix the ABI version of all libraries at 16.11.0.
This is a proven approach and has been used for years with different
libraries. You could easily do this independently of us upstream
fixing the ABI problems.


> 2. ABI Ranges

ABI is either backwards compatible (same major) or not. A range
doesn't solve the problem.

>
> 3. A lot of conflicts
>

This doesn't allow us to have multiple version of the library
available at runtime. So in the end it doesn't solve the problem for
the distro either.


>
> 4. ABI bump is infecting
>
> 5. back to single ABI
>

This is very similar to approach 1. It just uses up a lot more ABI versions.


> 6. More
> I'm sure there are more approaches to this, feel free to come up with more.
>

The problem is that we do not detect and fix the ABI changes that
"shine-through" the dependencies of our libraries. We need to work on
them and fix them one by one. Long-term we need to invest into keeping
the API/ABI stable and adding backward compatible symbols as well as
making structures opaque.



> I'm sure my five suggestions alone will make the thread messy, Maybe we do
> this in two rounds, sorting out the insane and identifying the preferred
> ones to then in a second run focus on discussing and maybe implementing the
> details of what we like.
>
>
> [1]: http://dpdk.org/browse/dpdk/tree/doc/guides/contributing/versioning.rst
> [2]: http://dpdk.org/browse/dpdk/commit/?id=d7e61ad3ae36
> [3]: http://dpdk.org/browse/dpdk/commit/?id=6ba1affa54108
> [4]: https://wiki.debian.org/TransitionBestPractices
> [5]: https://packages.debian.org/sid/libboost1.62-dev
> [6]:
> https://www.debian.org/doc/debian-policy/ch-relationships.html#s-conflicts
> [7]: https://wiki.ubuntu.com/ProposedMigration
>
> P.S. I beg a pardon for the wall of text
>
> --
> Christian Ehrhardt
> Software Engineer, Ubuntu Server
> Canonical Ltd