[dpdk-dev] vmware vmxnet3-usermap AND DPDK VMXNET3 PMD

[Patel, Rashmin N]

Please find comments in-lined.

Thanks,
RP

From: Aziz Hajee [mailto:aziz at saisei.com]
Sent: Monday, November 10, 2014 6:00 PM
To: Patel, Rashmin N
Cc: dev at dpdk.org
Subject: Re: [dpdk-dev] vmware vmxnet3-usermap AND DPDK VMXNET3 PMD

Rashmin,
Since I do need the jumbo, I use the vmxnet3-plugin you described, i.e.
(1)
sudo insmod ./vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=2048 num_txds=2048
and (2) when running the application, use in the args list:
"-d", "librte_pmd_vmxnet3.so"
Does the above two piece mean vmxnet3-plugin
[RP] that’s correct
I do see my vmxnet3 device from the dump,    rte_eal_pci_dump();
but the 'nb_ports' in DPDK never gets incremented rte_eth_dev_count() returns zero.
so all the other api fails, if (port_id >= nb_ports) {
                PMD_DEBUG_TRACE("Invalid port_id=%d\n", port_id);
                return;
        }

0000:03:00.0 - vendor:15ad device:7b0
   00000000d2404000 0000000000001000
   00000000d2403000 0000000000001000
   00000000d2400000 0000000000002000
   0000000000000000 0000000000000000
   0000000000000000 0000000000000000
   0000000000000000 0000000000000000
   00000000d4400000 0000000000010000
0000:0b:00.0 - vendor:15ad device:7b0
   00000000d2504000 0000000000001000
   00000000d2503000 0000000000001000
   00000000d2500000 0000000000002000
   0000000000000000 0000000000000000
   0000000000000000 0000000000000000
   0000000000000000 0000000000000000
   00000000d4500000 0000000000010000
DPDK: No Ethernet ports (rte_eth_dev_count() returns zero)
PMD: rte_eth_dev_info_get: Invalid port_id=0
PMD: rte_eth_dev_configure: Invalid port_id=0
PMD: rte_eth_dev_info_get: Invalid port_id=0
PMD: rte_eth_dev_configure: Invalid port_id=0
So when using not using DPDK PMD for VMXNET3, what am i missing, for the the DPDK to know the nb_ports,
How will the rte_eth_dev_start(portid) in DPDK library know  ?

rte_pmd_init_all() will not have the init the Intel DPDK PMD , RTE_LIBRTE_VMXNET3_PMD = n in config.
[RP] If you’re using the vmxnet3-plugin, you should keep RTE_LIBRTE_VMXNET3_PMD = n in config, and link the shared library with its headers, it should work. I have tried it once a long back.

#ifdef RTE_LIBRTE_VMXNET3_PMD
        if ((ret = rte_vmxnet3_pmd_init()) != 0) {
                RTE_LOG(ERR, PMD, "Cannot init vmxnet3 PMD\n");
                return (ret);
        }
If I make RTE_LIBRTE_VMXNET3_PMD = y, then I am using the Intel DPDK PMD and no jumbo.
[RP] Yes, I understood that part. We need to support jumbo frames in in-tree version of VMXNET3-PMD, we’ll merge all soon, we can discuss in the community conf. call so please do attend the next one on Nov 18 and we can raise concerns there

Thanks,
aziz

On Fri, Nov 7, 2014 at 8:53 AM, Patel, Rashmin N <rashmin.n.patel at intel.com<mailto:rashmin.n.patel at intel.com>> wrote:
Hi Aziz,

Yes, you're right DPDK VMXNET3-PMD in /lib/librte_pmd_vmxnet3 does not support mbuf chaining today. But it's a standalone bsd driver just like any other pmd in that directory, it does not need vmxnet3-usermap.ko module.

Now there is another vmxnet3 solution in a separate branch as a plugin, which must have vmxnet3-usermap.ko linux module(1), and a user space interface piece(2) to tie it to any DPDK application in the main branch. (1) and (2) makes the solution which is known as vmxnet3-plugin. It's been there for a long time just like virtio-plugin, I don't know who uses it, but community can *reply* here if there is still any need of a separate solution that way.

I'm in favor of consolidating all those version into one elegant solution by grabbing best features from all of them and maintain one copy. I'm sure that developers contributing from VMware would also support that idea because then it makes easy to maintain and debug and bug fix and most importantly avoid such confusion in future.

Thanks,
Rashmin

-----Original Message-----
From: dev [mailto:dev-bounces at dpdk.org<mailto:dev-bounces at dpdk.org>] On Behalf Of Aziz Hajee
Sent: Thursday, November 06, 2014 5:47 PM
To: dev at dpdk.org<mailto:dev at dpdk.org>
Subject: [dpdk-dev] vmware vmxnet3-usermap AND DPDK VMXNET3 PMD

I am using the dpdk1.6.0r1
I could not find a complete clarification, sorry if missed.
VMXNET3 PMD
============
I have enabled the VMXNET3 PMD  in the dpdk.
 # Compile burst-oriented VMXNET3 PMD driver  #

CONFIG_RTE_LIBRTE_VMXNET3_PMD=y
CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_INIT=y
CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_RX=n
CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX=n
CONFIG_RTE_LIBRTE_VMXNET3_DEBUG_TX_FREE=n
The Intel DPDK VMXNET3 PMD driver does not support mbuf chaining, and I have to set NOMULTSEGS for the vmxnet3 interface init to succeed.
tx_conf.txq_flags =  ETH_TXQ_FLAGS_NOMULTSEGS Is there a later version of DPDK that supports multiseg for the dpdk
VMXNET3 PMD.

vmware vmxnet3-usermap AND  DPDK VMXNET3 PMD =========================================
Is the vmxnet3-usermap.ko module driver also needed ? (appears that I need, otherwise the eal initialise fails.
sudo insmod ./vmxnet3-usermap.ko enable_shm=2,2 num_rqs=1,1 num_rxds=2048
num_txds=2048

I do not understand if VMXNET3 PMD is there, what is the purpose of /vmxnet3-usermap.ko/vmxnet3-usermap.ko ?

From some responses i saw that the following ifdef RTE_EAL_UNBIND_PORTS is also need to be removed in lib/librte_pmd_vmxnet3/vmxnet3_ethdev.c ?
     {
         .name = "rte_vmxnet3_pmd",
         .id_table = pci_id_vmxnet3_map, -#ifdef RTE_EAL_UNBIND_PORTS
+// #ifdef RTE_EAL_UNBIND_PORTS
         .drv_flags = RTE_PCI_DRV_NEED_IGB_UIO, -#endif
+// #endif
     },
     .eth_dev_init = eth_vmxnet3_dev_init,
     .dev_private_size = sizeof(struct vmxnet3_adapter),

thanks,
-aziz