Bug 537

Summary: FM1000 "PMD: eth_fm10k_dev_init(): switch is not ready" error
Product: DPDK Reporter: longyu (longyuwlz)
Component: ethdevAssignee: qi.z.zhang (qi.z.zhang)
Status: UNCONFIRMED ---    
Severity: major CC: ajit.khaparde, longyuwlz, qiming.yang, xiao.w.wang
Priority: High    
Version: 19.11   
Target Milestone: ---   
Hardware: x86   
OS: Linux   

Description longyu 2020-09-17 13:01:49 CEST
I am trying to run dpdk process on an fm10k NIC. I have configured dpdk to use fm10k driver,and the NIC is successfully recognized.

I have bound the fm10k interface to use igb_uio driver and tried to run l2fwd for testing.

I use the following command to run l2fwd:

./l2fwd -w 0000:05:00.0,enable_ftag=1 -w 0000:83:00.0,enable_ftag=1 -- -p 0x3

And sometimes the l2fwd gave me the following errors:

"PMD: eth_fm10k_dev_init(): switch is not ready"

I tried to add sleep and retry but it failed!
Comment 1 longyu 2020-09-17 13:04:36 CEST
add the command line output as the following:

[slot1] ./l2fwd -w 0000:05:00.0,enable_ftag=1 -w 0000:83:00.0,enable_ftag=1 -- -p 0x3
./l2fwd: /lib/libcrypto.so.10: no version information available (required by ./l2fwd)
EAL: Detected 20 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: Probing VFIO support...
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL:   probe driver: 8086:15a4 net_fm10k
eth_fm10k_dev_init(): switch is not ready
EAL: Requested device 0000:05:00.0 cannot be used
EAL: PCI device 0000:83:00.0 on NUMA socket 1
EAL:   probe driver: 8086:15a4 net_fm10k
EAL: set_mempolicy failed: Invalid argument
EAL: Requested device 0000:83:00.0 cannot be used
MAC updating enabled
EAL: Error - exiting with code: 1
  Cause: No Ethernet ports - bye
Comment 2 Ajit Khaparde 2020-09-18 22:37:38 CEST
Please take a look. Thanks
Comment 3 longyu 2020-09-22 10:19:22 CEST
dpdk version: 16.04 / 19.11
switch kernel version : Linux localhost.localdomain 3.17.4-301.fc21.x86_64 #1 SMP Thu Nov 27 19:09:10 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Work board kernel version: Linux cneos 3.16.35-nsfocus.x86_64 #1 SMP PREEMPT Fri Aug 7 12:41:32 CST 2020 x86_64 GNU/Linux

The following information hopes to be helpful for you:

I found this problem on dpdk-16.04 first. When I run l2fwd(dpdk-16.04) on work board for one day, I rebooted the work board many times, and then I run l2fwd again, now it reported an error : “PMD: eth_fm10k_dev_init(): default VID is not ready” . 

The command line is like the followings:

“L2fwd -w 0000:05:00.,enable_ftag=1 -w 0000:83:00.0,enable_ftag=1 -- -p 0x3”.

I have tried to modify the fm10k init code like this :

/* Make sure default VID is ready before going forward. */
        if (hw->mac.type == fm10k_mac_pf) {
                for (i = 0; i < MAX_QUERY_SWITCH_STATE_TIMES; i++) {
                        printf("default vid is ;%u\n",hw->mac.default_vid);
                        if (hw->mac.default_vid)
                                break;
                        /* Delay some time to acquire async port VLAN info. */
                        rte_delay_us(WAIT_SWITCH_MSG_US);
                }

                if (!hw->mac.default_vid) {
                        hw->mac.default_vid = 1;
                        /* PMD_INIT_LOG(ERR, "default VID is not ready");
                        return -1;
*/
                }
        }
And then, I run the l2fwd, this time, it reported an new error:”PMD: eth_fm10k_dev_init(): switch is not ready”.

I have tried to insmod fm10k.ko on work board, and bind NIC port to fm10k driver. After that I run “ifconfig enp5s0 up” to up the fm10k interface, and run ethtool,the output like this:

[slot2] ifconfig enp5s0 up
[slot2] ethtool enp5s0
Settings for enp5s0:
        	Current message level: 0x00000007 (7)
                               drv probe link
        	Link detected: no

I found the fm10k port couldn’t up. Then, I reseted(ifconfig down/up) the fm10k interface on switch, and ifconfig down/up the fm10k interface on work board once again , it succeeded. And then, I run the l2fwd, it still reported error: “PMD: eth_fm10k_dev_init(): switch is not ready”.

When I run l2fwd on work board, I have tried to reset(ifconfig down/up) the switch fm10k interface again and again. At this time, the l2fwd initialized successfully.

When I rebooted the switch, l2fwd worked fine.

I checked the fm10k.ko version about switch and work board:

work board fm10k.ko version: 0.27.1
switch fm10k.ko version: 0.23.1
Comment 4 Qimingy 2020-12-22 07:56:29 CET
have you try latest DPDK code? what's your FW and kernel driver version?