[dpdk-dev] [PATCH] Fix linking errors when CONFIG_RTE_BUILD_SHARED_LIB is enabled

[Sergio Gonzalez Monroy]

On Thu, Oct 02, 2014 at 05:28:04PM +0900, Tetsuya Mukawa wrote:
> (2014/10/02 17:12), Sergio Gonzalez Monroy wrote:
> > On Thu, Oct 02, 2014 at 11:48:37AM +0900, Tetsuya Mukawa wrote:
> >
> >> $ gcc -m64 -pthread -fPIC -march=native -DRTE_MACHINE_CPUFLAG_SSE
> >> -DRTE_MACHINE_CPUFLAG_SSE2 -DRTE_MACHINE_CPUFLAG_SSE3
> >> -DRTE_MACHINE_CPUFLAG_SSSE3 -DRTE_MACHINE_CPUFLAG_SSE4_1
> >> -DRTE_MACHINE_CPUFLAG_SSE4_2 -DRTE_MACHINE_CPUFLAG_AES
> >> -DRTE_MACHINE_CPUFLAG_PCLMULQDQ -DRTE_MACHINE_CPUFLAG_AVX
> >> -DRTE_MACHINE_CPUFLAG_F16C
> >> -DRTE_COMPILE_TIME_CPUFLAGS=RTE_CPUFLAG_SSE,RTE_CPUFLAG_SSE2,RTE_CPUFLAG_SSE3,RTE_CPUFLAG_SSSE3,RTE_CPUFLAG_SSE4_1,RTE_CPUFLAG_SSE4_2,RTE_CPUFLAG_AES,RTE_CPUFLAG_PCLMULQDQ,RTE_CPUFLAG_AVX,RTE_CPUFLAG_F16C
> >> -I/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include -include
> >> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include/rte_config.h
> >> -W -Wall -Werror -Wstrict-prototypes -Wmissing-prototypes
> >> -Wmissing-declarations -Wold-style-definition -Wpointer-arith
> >> -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral
> >> -Wformat-security -Wundef -Wwrite-strings -Wl,-Map=testacl.map,--cref
> >> -o testacl main.o -Wl,-export-dynamic
> >> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> >> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> >> -Wl,--whole-archive -Wl,-lrte_distributor -Wl,-lrte_kni
> >> -Wl,-lrte_pipeline -Wl,-lrte_table -Wl,-lrte_port -Wl,-lrte_timer
> >> -Wl,-lrte_hash -Wl,-lrte_lpm -Wl,-lrte_power -Wl,-lrte_acl
> >> -Wl,-lrte_meter -Wl,-lrte_sched -Wl,-lm -Wl,-lrt -Wl,--start-group
> >> -Wl,-lrte_kvargs -Wl,-lrte_mbuf -Wl,-lrte_ip_frag -Wl,-lethdev
> >> -Wl,-lrte_malloc -Wl,-lrte_mempool -Wl,-lrte_ring -Wl,-lrte_eal
> >> -Wl,-lrte_cmdline -Wl,-lrte_cfgfile -Wl,-lrte_pmd_bond -Wl,-lrt
> >> -Wl,-lm -Wl,-lgcc_s -Wl,-ldl -Wl,--end-group -Wl,--no-whole-archive
> >> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> >> undefined reference to `rte_mempool_lookup'
> >> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> >> undefined reference to `rte_mempool_create' collect2: error: ld
> >> returned 1 exit status 
> > Hi Tetsuya,
> >
> > Would you mind posting the output of the last command adding the option '-v'?
> Sure, here is.
> 
> $ gcc -m64 -pthread -fPIC  -march=native -DRTE_MACHINE_CPUFLAG_SSE
> -DRTE_MACHINE_CPUFLAG_SSE2 -DRTE_MACHINE_CPUFLAG_SSE3
> -DRTE_MACHINE_CPUFLAG_SSSE3 -DRTE_MACHINE_CPUFLAG_SSE4_1
> -DRTE_MACHINE_CPUFLAG_SSE4_2 -DRTE_MACHINE_CPUFLAG_AES
> -DRTE_MACHINE_CPUFLAG_PCLMULQDQ -DRTE_MACHINE_CPUFLAG_AVX
> -DRTE_MACHINE_CPUFLAG_F16C
> -DRTE_COMPILE_TIME_CPUFLAGS=RTE_CPUFLAG_SSE,RTE_CPUFLAG_SSE2,RTE_CPUFLAG_SSE3,RTE_CPUFLAG_SSSE3,RTE_CPUFLAG_SSE4_1,RTE_CPUFLAG_SSE4_2,RTE_CPUFLAG_AES,RTE_CPUFLAG_PCLMULQDQ,RTE_CPUFLAG_AVX,RTE_CPUFLAG_F16C 
> -I/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include -include
> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include/rte_config.h -W
> -Wall -Werror -Wstrict-prototypes -Wmissing-prototypes
> -Wmissing-declarations -Wold-style-definition -Wpointer-arith
> -Wcast-align -Wnested-externs -Wcast-qual -Wformat-nonliteral
> -Wformat-security -Wundef -Wwrite-strings  -Wl,-Map=testacl.map,--cref
> -o testacl main.o -Wl,-export-dynamic
> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib 
> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> -Wl,--whole-archive -Wl,-lrte_distributor -Wl,-lrte_kni
> -Wl,-lrte_pipeline -Wl,-lrte_table -Wl,-lrte_port -Wl,-lrte_timer
> -Wl,-lrte_hash -Wl,-lrte_lpm -Wl,-lrte_power -Wl,-lrte_acl
> -Wl,-lrte_meter -Wl,-lrte_sched -Wl,-lm -Wl,-lrt -Wl,--start-group
> -Wl,-lrte_kvargs -Wl,-lrte_mbuf -Wl,-lrte_ip_frag -Wl,-lethdev
> -Wl,-lrte_malloc -Wl,-lrte_mempool -Wl,-lrte_ring -Wl,-lrte_eal
> -Wl,-lrte_cmdline -Wl,-lrte_cfgfile -Wl,-lrte_pmd_bond -Wl,-lrt -Wl,-lm
> -Wl,-lgcc_s -Wl,-ldl -Wl,--end-group -Wl,--no-whole-archive -v
> Using built-in specs.
> COLLECT_GCC=gcc
> COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Ubuntu
> 4.8.2-19ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs
> --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr
> --program-suffix=-4.8 --enable-shared --enable-linker-build-id
> --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix
> --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib
> --enable-nls --with-sysroot=/ --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-libstdcxx-time=yes
> --enable-gnu-unique-object --disable-libmudflap --enable-plugin
> --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk
> --enable-gtk-cairo
> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre
> --enable-java-home
> --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64
> --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64
> --with-arch-directory=amd64
> --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc
> --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64
> --with-multilib-list=m32,m64,mx32 --with-tune=generic
> --enable-checking=release --build=x86_64-linux-gnu
> --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1)
> COMPILER_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/
> LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/4.8/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib/:/lib/x86_64-linux-gnu/:/lib/../lib/:/usr/lib/x86_64-linux-gnu/:/usr/lib/../lib/:/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../:/lib/:/usr/lib/
> COLLECT_GCC_OPTIONS='-m64' '-pthread' '-fPIC' '-march=native' '-D'
> 'RTE_MACHINE_CPUFLAG_SSE' '-D' 'RTE_MACHINE_CPUFLAG_SSE2' '-D'
> 'RTE_MACHINE_CPUFLAG_SSE3' '-D' 'RTE_MACHINE_CPUFLAG_SSSE3' '-D'
> 'RTE_MACHINE_CPUFLAG_SSE4_1' '-D' 'RTE_MACHINE_CPUFLAG_SSE4_2' '-D'
> 'RTE_MACHINE_CPUFLAG_AES' '-D' 'RTE_MACHINE_CPUFLAG_PCLMULQDQ' '-D'
> 'RTE_MACHINE_CPUFLAG_AVX' '-D' 'RTE_MACHINE_CPUFLAG_F16C' '-D'
> 'RTE_COMPILE_TIME_CPUFLAGS=RTE_CPUFLAG_SSE,RTE_CPUFLAG_SSE2,RTE_CPUFLAG_SSE3,RTE_CPUFLAG_SSSE3,RTE_CPUFLAG_SSE4_1,RTE_CPUFLAG_SSE4_2,RTE_CPUFLAG_AES,RTE_CPUFLAG_PCLMULQDQ,RTE_CPUFLAG_AVX,RTE_CPUFLAG_F16C'
> '-I' '/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include'
> '-include'
> '/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/include/rte_config.h'
> '-Wextra' '-Wall' '-Werror' '-Wstrict-prototypes' '-Wmissing-prototypes'
> '-Wmissing-declarations' '-Wold-style-definition' '-Wpointer-arith'
> '-Wcast-align' '-Wnested-externs' '-Wcast-qual' '-Wformat-nonliteral'
> '-Wformat-security' '-Wundef' '-Wwrite-strings' '-o' 'testacl'
> '-L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib'
> '-L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib' '-v'

We have been looking at this issue over the last couple of days.
It seems to be an Ubuntu GCC related bug as it works on other distros.
GCC is passing incorrectly the option '--as-needed' to LD resulting in wrong linking.

>  /usr/lib/gcc/x86_64-linux-gnu/4.8/collect2 --sysroot=/ --build-id
> --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed

The '--as-needed' just at the end of the above line is incorrect and not matching
a closing '--no-as-needed'.
You may get away with it by adding 'EXTRA_LDFLAGS=--no-as-needed'. 
ie. $ make install T=x86_64-native-linuxapp-gcc EXTRA_LDFLAGS=--no-as-needed

Sergio

> -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o testacl
> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o
> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crti.o
> /usr/lib/gcc/x86_64-linux-gnu/4.8/crtbegin.o
> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> -L/home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib
> -L/usr/lib/gcc/x86_64-linux-gnu/4.8
> -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu
> -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib
> -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu
> -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../..
> -Map=testacl.map --cref main.o -export-dynamic --whole-archive
> -lrte_distributor -lrte_kni -lrte_pipeline -lrte_table -lrte_port
> -lrte_timer -lrte_hash -lrte_lpm -lrte_power -lrte_acl -lrte_meter
> -lrte_sched -lm -lrt --start-group -lrte_kvargs -lrte_mbuf -lrte_ip_frag
> -lethdev -lrte_malloc -lrte_mempool -lrte_ring -lrte_eal -lrte_cmdline
> -lrte_cfgfile -lrte_pmd_bond -lrt -lm -lgcc_s -ldl --end-group
> --no-whole-archive -lgcc --as-needed -lgcc_s --no-as-needed -lpthread
> -lc -lgcc --as-needed -lgcc_s --no-as-needed
> /usr/lib/gcc/x86_64-linux-gnu/4.8/crtend.o
> /usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crtn.o
> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> undefined reference to `rte_mempool_lookup'
> /home/mukawa/tmp/dpdk/x86_64-native-linuxapp-gcc/lib/librte_eal.so:
> undefined reference to `rte_mempool_create'
> collect2: error: ld returned 1 exit status
> 
> Regards,
> Tetsuya
> 
> > Sergio
> >
> >
> >> ----------------------------------------------------------------------
> >>
> >> I just enabled the following option, then compile DPDK.
> >> CONFIG_RTE_BUILD_SHARED_LIB
> >> Is this collect to compile PMDs as dynamic link libraries?
> >>
> >> With the option, all libraries include are compiled as dynamic link library.
> >> Does "--start-group/--end-group" options work with dynamic link libraries?
> >>
> >> Regards,
> >> Tetsuya
> >>
>