Bug 27 - Can't build mlx5 from 18.02 on CentOS Linux release 7.4.1708 with kernel 3.10.0-693.11.1.el7.x86_64
Summary: Can't build mlx5 from 18.02 on CentOS Linux release 7.4.1708 with kernel 3.10...
Status: RESOLVED INVALID
Alias: None
Product: DPDK
Classification: Unclassified
Component: core (show other bugs)
Version: 18.02
Hardware: x86 Linux
: Normal major
Target Milestone: ---
Assignee: Yongseok Koh
URL:
Depends on:
Blocks:
 
Reported: 2018-04-20 15:58 CEST by bearrailgun
Modified: 2019-10-25 09:28 CEST (History)
3 users (show)



Attachments

Description bearrailgun 2018-04-20 15:58:38 CEST
Release notes for 18.02 say it was tested on Red Hat Enterprise Linux Server release 7.4 with MLNX_OFED: 4.3-0.1.6.0.

I installed mellanox ofed 4.3-1.0.1.0 ( a little bit newer but still).
Building 18.02 release with CONFIG_RTE_BUILD_SHARED_LIB=y, CONFIG_RTE_LIBRTE_MLX5_PMD=y, CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS=y.
Other options left as default.

Anyway, when its going to compile drivers/net/mlx5 I have next errors:

== Build drivers/net/mlx5
  CC mlx5.o
In file included from /root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:42:0:
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:64:11: warning: ‘struct ibv_rwq_ind_table_init_attr’ declared inside parameter list [enabled by default]
    struct ibv_rwq_ind_table_init_attr *init_attr);
           ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:64:11: warning: its scope is only this definition or declaration, which is probably not what you want [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:67:16: warning: ‘struct ibv_wq_init_attr’ declared inside parameter list [enabled by default]
         struct ibv_wq_init_attr *wq_init_attr);
                ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:69:45: warning: ‘struct ibv_wq_attr’ declared inside parameter list [enabled by default]
  int (*modify_wq)(struct ibv_wq *wq, struct ibv_wq_attr *wq_attr);
                                             ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:98:39: warning: ‘struct ibv_cq_ex’ declared inside parameter list [enabled by default]
  struct ibv_cq *(*cq_ex_to_cq)(struct ibv_cq_ex *cq);
                                       ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:102:11: warning: ‘struct mlx5dv_cq_init_attr’ declared inside parameter list [enabled by default]
    struct mlx5dv_cq_init_attr *mlx5_cq_attr);
           ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:102:11: warning: ‘struct ibv_cq_init_attr_ex’ declared inside parameter list [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5_glue.h:107:8: warning: ‘enum mlx5dv_set_ctx_attr_type’ declared inside parameter list [enabled by default]
        void *attr);
        ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c: In function ‘mlx5_pci_probe’:
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:837:37: error: ‘struct ibv_device_attr_ex’ has no member named ‘device_cap_flags_ex’
   config.hw_csum = !!(device_attr_ex.device_cap_flags_ex &
                                     ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:838:9: error: ‘IBV_DEVICE_RAW_IP_CSUM’ undeclared (first use in this function)
         IBV_DEVICE_RAW_IP_CSUM);
         ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:838:9: note: each undeclared identifier is reported only once for each function it appears in
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:858:18: error: ‘struct ibv_device_attr_ex’ has no member named ‘rss_caps’
    device_attr_ex.rss_caps.max_rwq_indirection_table_size;
                  ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:866:43: error: ‘struct ibv_device_attr_ex’ has no member named ‘raw_packet_caps’
   config.hw_vlan_strip = !!(device_attr_ex.raw_packet_caps &
                                           ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:867:7: error: ‘IBV_RAW_PACKET_CAP_CVLAN_STRIPPING’ undeclared (first use in this function)
       IBV_RAW_PACKET_CAP_CVLAN_STRIPPING);
       ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:871:42: error: ‘struct ibv_device_attr_ex’ has no member named ‘raw_packet_caps’
   config.hw_fcs_strip = !!(device_attr_ex.raw_packet_caps &
                                          ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:872:7: error: ‘IBV_RAW_PACKET_CAP_SCATTER_FCS’ undeclared (first use in this function)
       IBV_RAW_PACKET_CAP_SCATTER_FCS);
       ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:884:32: error: ‘struct ibv_device_attr_ex’ has no member named ‘tso_caps’
   config.tso = ((device_attr_ex.tso_caps.max_tso > 0) &&
                                ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:885:25: error: ‘struct ibv_device_attr_ex’ has no member named ‘tso_caps’
          (device_attr_ex.tso_caps.supported_qpts &
                         ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:889:20: error: ‘struct ibv_device_attr_ex’ has no member named ‘tso_caps’
      device_attr_ex.tso_caps.max_tso;
                    ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:957:10: error: variable ‘alctr’ has initializer but incomplete type
   struct mlx5dv_ctx_allocators alctr = {
          ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:958:4: error: unknown field ‘alloc’ specified in initializer
    .alloc = &mlx5_alloc_verbs_buf,
    ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:958:4: warning: excess elements in struct initializer [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:958:4: warning: (near initialization for ‘alctr’) [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:959:4: error: unknown field ‘free’ specified in initializer
    .free = &mlx5_free_verbs_buf,
    ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:959:4: warning: excess elements in struct initializer [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:959:4: warning: (near initialization for ‘alctr’) [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:960:4: error: unknown field ‘data’ specified in initializer
    .data = priv,
    ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:960:4: warning: excess elements in struct initializer [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:960:4: warning: (near initialization for ‘alctr’) [enabled by default]
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:957:32: error: storage size of ‘alctr’ isn’t known
   struct mlx5dv_ctx_allocators alctr = {
                                ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:963:13: error: ‘MLX5DV_CTX_ATTR_BUF_ALLOCATORS’ undeclared (first use in this function)
             MLX5DV_CTX_ATTR_BUF_ALLOCATORS,
             ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:964:13: error: type of formal parameter 2 is incomplete
             (void *)((uintptr_t)&alctr));
             ^
/root/dpdk/dpdk-18.02/drivers/net/mlx5/mlx5.c:957:32: warning: unused variable ‘alctr’ [-Wunused-variable]
   struct mlx5dv_ctx_allocators alctr = {
                                ^
make[6]: *** [mlx5.o] Error 1
make[5]: *** [mlx5] Error 2
make[4]: *** [net] Error 2
make[3]: *** [drivers] Error 2
make[2]: *** [all] Error 2
make[1]: *** [pre_install] Error 2
make: *** [install] Error 2


kernel version is 3.10.0-693.11.1.el7.x86_64

what is wrong?
Comment 1 Brian Nesbitt 2018-05-03 17:05:01 CEST
Hi,

I had same issue with Red Hat 7.4 and CentOS 7.4. I managed to get around this by uninstalling all the official MLX software and drivers and just using the Red Hat rdma-core-devel (and dependent) packages. This RPM provides the newer verbs.h and MLX headers.

It might be useful to raise this with Mellanox as their own software appears to be incompatible with dpdk >= 17.x.

Regards
Brian.
Comment 2 Ajit Khaparde 2018-08-02 22:40:51 CEST
Can you please take a look and take to closure.

Thanks
Comment 3 Yongseok Koh 2018-08-02 22:50:02 CEST
Brian is right.

When you install MLNX_OFED, you should specify the following options in order to install upstream rdma-core library.
    # ./mlnxofedinstall --upstream-libs --dpdk

It is documented in 24.8. Quick Start Guide on OFED
(http://doc.dpdk.org/guides/nics/mlx5.html)

Thanks,
Yongseok
Comment 4 Matt Rundle 2019-10-25 08:54:31 CEST
Later versions of DPDK contain references to, amongst others, MLX5DV_CTX_ATTR_BUF_ALLOCATORS.

I've scanned through all of the latest OFED versions (4.5-4.7), including 4.5 and 4.6 which is recommended in the DPDK documentation, and none of the provided mlnx5dv.h files contain the definition of this enum.

What am I missing here?
Comment 5 Matt Rundle 2019-10-25 09:28:34 CEST
For more detail, I downloaded MLNX_OFED_LINUX-4.7-1.0.0.1-rhel8.1-x86_64.tgz and then unpacked all of the *.x86_64.rpm files within it via `rpm2cpio $rpm | cpio -idmv`. Grepping for CTX_ATTR_BUF yields a definition for mlx4dv, but not mlx5dv:    

    $ basename `pwd`
    MLNX_OFED_LINUX-4.7-1.0.0.1-rhel8.1-x86_64
    
    $ grep -r CTX_ATTR_BUF all_rpm_extracts/usr/include/
all_rpm_extracts/usr/include/infiniband/mlx4dv.h:	MLX4DV_SET_CTX_ATTR_BUF_ALLOCATORS	= 1,


But MLX5DV_CTX_ATTR_BUF_ALLOCATORS is used in mlx5.c: https://git.dpdk.org/dpdk/tree/drivers/net/mlx5/mlx5.c?h=v18.02#n963 (v18.02)    

Setting to unconfirmed as this appears to be a legitimate issue.

Note You need to log in before you can comment on or make changes to this bug.