[dpdk-dev] [VMXNET3]: device activation failed, while hotplug a vmxnet3 nic

[Ke Xu(许可)]

Hi,

I hit an interesting issue recently, while I try to hotplug a vmxnet3 nic to a vm.
Here is the environment,
         ESXi 6.0
         DPDK-16.07
         Guest linux kernel 3.10.20

VM is running in ESXi6.0, I use Vsphere Client to Edit setting,  add a vmxnet3 nic to this running vm (hotplug).  Device is failed to start.
In vmxnet3_dev_start(),
         /* Activate device by register write */
         VMXNET3_WRITE_BAR1_REG(hw, VMXNET3_REG_CMD, VMXNET3_CMD_ACTIVATE_DEV);
         status = VMXNET3_READ_BAR1_REG(hw, VMXNET3_REG_CMD);

         if (status != 0) {
                   PMD_INIT_LOG(ERR, "Device activation: UNSUCCESSFUL");
                   return -1;
         }
Also,  I have these error log:
In /var/log/vmkernel.log
         …
         2017-12-07T14:00:37.524Z cpu18:32859)Net: 3649: disconnected client from port 0x20005f9
2017-12-07T14:00:38.559Z cpu19:4357909)Vmxnet3: 12840: invalid intr index 56 for tq 0, only 1 intr allocated
…
In /vmfs/volumes/57073409-d705da7d-8b7c-44a8422585c2/KFC_8373/vmware.log
         …
         2017-12-07T14:00:38.559Z| vcpu-0| I120: VMXNET3 user: failed to activate Ethernet1, status: 0xbad0001
         ..

I also notice these llinks,
         https://kb.vmware.com/s/article/2136932                                “ESXi 5.5 Update 3 and 6.0 permits only a maximum Rx Ring #2 size of 2048”
         https://mail-archive.com/dev@dpdk.org/msg14401.html    “Vmxnet3 activation of device fails in DPDK1.7”

in my envirment, rx desc num is 1024, as same as tx desc num.  I’m not sure if it is the root case.

For more information,  vmxnet3 nics are OK while VM is cold boot. ( nic is not hotplugged )

Anyone met this issue before? It seems like interrupt problem, but I m not sure it is vmxnet3 pmd driver issue or igb uio issue.
I’m glad to have your reply.

Best regards,
XU Ke