[dpdk-dev] Inter-PMD dependencies when building shared libraries

[Olivier MATZ]

Hi Gage,

+CC Thomas, Bruce

On Mon, Oct 02, 2017 at 08:48:29PM +0000, Eads, Gage wrote:
> I believe I've spotted an issue in the way inter-PMD dependencies are handled when building shared libraries. The depdirs_rule in mk/rte.subdir.mk relies on DEPDIRS-xyz containing the names of subdirectories that xyz depends on. In mk/rte.lib.mk, these DEPDIRS are converted into LDLIBS. This works when the subdirectory names match the library names (i.e. any of the libraries under lib/). However when the dependency is on a PMD, the subdirectory and library names don't match.
> 
> This is a problem, for example, in a patch for the net/octeontx PMD, which has a dependency on the event/octeontx PMD: http://dpdk.org/ml/archives/dev/2017-August/073983.html
> 
> I've reproduced this with a contrived example, by making the failsafe PMD depend on the NULL PMD in drivers/net/Makefile:
> -DEPDIRS-failsafe = $(core-libs)
> +DEPDIRS-failsafe = $(core-libs) librte_pmd_null
> 
> You can reproduce the build failure by running this command:
> ./devtools/test-build.sh x86_64-native-linuxapp-gcc+CONFIG_RTE_BUILD_SHARED_LIB
> 
> I'm no expert on DPDK's dependency handling code, but one option is to modify rte.lib.mk like so:
> -LDLIBS += $(subst lib,-l,$(_LDDIRS))
> +LDLIBS += $(subst lib,-l,$(filter lib%,$(_LDDIRS)))
> 
> Then you could put the PMD's directory name in DEPDIRs, and specify the depended-on library in the PMD's LDLIBS (as is done in the aforementioned net/octeontx PMD).
> 
> Thoughts?

Thanks for spotting this. Indeed, generating the "-l lib" LDFLAG from
DEPDIR doesn't work for PMDs. Here is the list of libraries whose directory
name is different than library name:

$ git grep "LIB = " | while read line; do dir=$(basename ${line%/Makefile*}); lib=${line#*= }; lib=${lib%*.a}; if [ "$dir" != "$lib" ]; then echo "${dir} ${lib}"; fi; done
fslmc librte_bus_fslmc
aesni_gcm librte_pmd_aesni_gcm
aesni_mb librte_pmd_aesni_mb
armv8 librte_pmd_armv8
dpaa2_sec librte_pmd_dpaa2_sec
kasumi librte_pmd_kasumi
null librte_pmd_null_crypto
openssl librte_pmd_openssl
qat librte_pmd_qat
scheduler librte_pmd_crypto_scheduler
snow3g librte_pmd_snow3g
zuc librte_pmd_zuc
dpaa2 librte_pmd_dpaa2_event
octeontx librte_pmd_octeontx_ssovf
skeleton librte_pmd_skeleton_event
sw librte_pmd_sw_event
dpaa2 librte_mempool_dpaa2
ring librte_mempool_ring
stack librte_mempool_stack
af_packet librte_pmd_af_packet
ark librte_pmd_ark
avp librte_pmd_avp
bnx2x librte_pmd_bnx2x
bnxt librte_pmd_bnxt
bonding librte_pmd_bond
cxgbe librte_pmd_cxgbe
dpaa2 librte_pmd_dpaa2
e1000 librte_pmd_e1000
ena librte_pmd_ena
enic librte_pmd_enic
failsafe librte_pmd_failsafe
fm10k librte_pmd_fm10k
i40e librte_pmd_i40e
ixgbe librte_pmd_ixgbe
kni librte_pmd_kni
liquidio librte_pmd_lio
mlx4 librte_pmd_mlx4
mlx5 librte_pmd_mlx5
nfp librte_pmd_nfp
null librte_pmd_null
pcap librte_pmd_pcap
qede librte_pmd_qede
ring librte_pmd_ring
sfc librte_pmd_sfc_efx
szedata2 librte_pmd_szedata2
tap librte_pmd_tap
thunderx librte_pmd_thunderx_nicvf
vhost librte_pmd_vhost
virtio librte_pmd_virtio
vmxnet3 librte_pmd_vmxnet3_uio
xenvirt librte_pmd_xenvirt
lib librte_ethtool
eal librte_eal
librte_ether librte_ethdev

If we don't consider the ones where we can add "librte_pmd_", the list is
smaller, but there are still many:

$ git grep "LIB = " | while read line; do dir=$(basename ${line%/Makefile*}); lib=${line#*= }; lib=${lib%*.a}; if [ "$dir" != "$lib" -a "librte_pmd_$dir" != "$lib" ]; then echo "${dir} ${lib}"; fi; done
fslmc librte_bus_fslmc
null librte_pmd_null_crypto
scheduler librte_pmd_crypto_scheduler
dpaa2 librte_pmd_dpaa2_event
octeontx librte_pmd_octeontx_ssovf
skeleton librte_pmd_skeleton_event
sw librte_pmd_sw_event
dpaa2 librte_mempool_dpaa2
ring librte_mempool_ring
stack librte_mempool_stack
bonding librte_pmd_bond
liquidio librte_pmd_lio
sfc librte_pmd_sfc_efx
thunderx librte_pmd_thunderx_nicvf
vmxnet3 librte_pmd_vmxnet3_uio
lib librte_ethtool
eal librte_eal
librte_ether librte_ethdev


To solve this, we can either:

1- do additional fixes of the "dir name -> lib name" conversion in
   lib.mk for all this list above, as we are doing for eal or
   ethdev. This looks feasible, but quite ugly.

2- rename all directories to match the libname (ex: drivers/net/null
   becomes drivers/net/librte_pmd_null). Looks to be a bad idea ;)

3- stop to generate the list of libraries from depdirs: DEPDIRS is
   kept for directory dependency at build, and the list of libraries
   is advertised in LDLIBS variable, in each Makefile.

My vote would go for option 3.