[dpdk-dev] How to manage new APIs added after major ABI release?

[Bruce Richardson]

On Wed, Dec 11, 2019 at 03:46:10PM +0000, Ferruh Yigit wrote:
> On 12/11/2019 3:02 PM, Thomas Monjalon wrote:
> > 11/12/2019 14:30, Ferruh Yigit:
> >> On 12/11/2019 1:11 PM, Neil Horman wrote:
> >>> On Tue, Dec 10, 2019 at 11:56:28AM +0000, Ferruh Yigit wrote:
> >>>> Hi,
> >>>>
> >>>> With new process, the major ABI releases will be compatible until it is
> >>>> deprecated (until next LTS for now),
> >>>> like current ABI version is 20 in DPDK_19.11 and DPDK versions until DPDK_20.11
> >>>> will be ABI compatible with this version.
> >>>>
> >>>> But if we introduce a new API after major ABI, say in 20.02 release, are we
> >>>> allowed to break the ABI for that API before DPDK_20.11?
> >>>>
> >>>> If we allow it break, following problem will be observed:
> >>>> Assume an application using .so.20.1 library, and using the new API introduced
> >>>> in 20.02, lets say foo(),
> >>>> but when application switches to .so.20.2 (released via DPDK_20.05), application
> >>>> will fail because of ABI breakage in foo().
> >>>>
> >>>> I think it is fair that application expects forward compatibility in minor
> >>>> versions of a shared library.
> >>>> Like if application linked against .so.20.2, fair to expect .so.20.3, .so.20.4
> >>>> etc will work fine. I think currently only .so.20.0 is fully forward compatible.
> >>>>
> >>>> If we all agree on this, we may need to tweak the process a little, but before
> >>>> diving into implementation details, I would like to be sure we are in same page.
> >>>>
> >>> Yes, I agree with the assertion.  Once an ABI is fixed, it must be compatible
> >>> with all future minor releases subsequent to the fixing of that ABI, until the
> >>> next major update.  That is to say, once you release ABI_20, all minor updates
> >>> 20.01, 20.02, etc must be compatible with ABI_20 until such time as ABI_21 is
> >>> released.
> >>
> >> There is a slight difference. All minor versions already compatible with ABI_20,
> >> like: 20.01, 20.02, 20.03 are ABI compatible with 20.0 (which defines ABI_20).
> >>
> >> Question is if 20.03 should be compatible with 20.02?
> >>
> >> This can happen if a new API is introduced in 20.2 and ABI has broken for that
> >> API in 20.3, so an ABI compatibility issue created between 20.03 & 20.02,
> >> meanwhile both are compatible with ABI_20.
> >>
> >> I can see two options:
> >> a) New APIs are introduced only when we switch to new major ABI version. But if
> >> we switch to longer (2 years) ABI compatibility, I think this is unacceptable to
> >> wait up to two years to have (non experimental) APIs.
> > 
> > I agree we should allow to add a new stable API in the middle of an ABI lifecycle.
> > 
> >> b) APIs added in minor version will be part of ABI_20 after that point and same
> >> rules will apply to them. Like if and API has introduced in 20.2, it is not
> >> allowed to be broken until next major ABI version.
> > 
> > Yes I think it is compliant with the agreed policy.
> 
> So I think two minor changes are required in the process to reflect this,
> 1) In ABI policy [1], it mentions in minor release both ABI_20 and ABI_21 can
> exist together, also in graph it says for minor versions:
> "v21 symbols are added and v20 symbols are modified, support for v20 ABI continues."
> I am not sure if we can call the symbols added in minor versions as v21 ABI,
> because it implies ABI compatibility is not required for them.
> 
> 2) In ABI versioning [2], documented as .map files will have 'DPDK_20' and
> 'DPDK_21' blocks.
> But instead, I think they should have 'DPDK_20.0', 'DPDK_20.1', ... blocks, and
> when major ABI version changed they all can be flattened to 'DPDK_21.0'.
> For example we can't do ABI versioning between 20.2 & 20.3 if we don't have
> these blocks.
> Current block names in .map files are already defined as 'DPDK_20.0', what we
> need to do is update the document to use 'DPDK_20.x' for the symbols added in
> minor version and follow that process.
> 

What do we really gain from making such a change from the policy? I think
it will work fine as-is, with putting all new symbols in the DPDK_21
section. Whatever way you look at it, the functions will be forward but not
backward compatible, which is all that really matters.

/Bruce