[dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF

[Dai, Wei]

> -----Original Message-----
> From: Wu, Yanglong
> Sent: Friday, January 12, 2018 10:38 AM
> To: Dai, Wei <wei.dai at intel.com>; dev at dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF
> 
> Hi
> After confirm with Daiwei and according to the data sheet,
> RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool is the Max number of queue can
> be  used in VF, that would be assigned as IXGBE_MAX_RX_QUEUE_NUM /
> RTE_ETH_DEV_SRIOV(dev).active, so that assigning nb_q_per_pool  as 1
> when PF is in ETH_MQ_RX_NONE, which will make VF can just use only 1
> queue, is not right. So this patch is right.
> 
> Yanglong
> 
> -----Original Message-----
> From: Wu, Yanglong
> Sent: Thursday, January 11, 2018 11:29 PM
> To: Wu, Yanglong <yanglong.wu at intel.com>; Dai, Wei
> <wei.dai at intel.com>; dev at dpdk.org
> Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF
> 
> Hi daiwei:
> 
> After I double checked with this bug, the problem can be described as follow:
> When we start PF using testpmd like this: testpmd -c 0x3 -n 4
> --socket-mem=1024,1024 --file-prefix=pf -w 0000:05:00.0 -w 0000:05:00.1
> -- -i. The number of queue signed to each port would be 1 and the
> rxmode.mq_mode of ixgbe eth_dev  would be set as  ETH_MQ_RX_NONE.
> And this config info is make sense, so until now there is no problem, right?
> And according to the code:
> 		case ETH_MQ_RX_NONE:
> >  			/* if nothing mq mode configure, use default scheme */
> >  			dev->data->dev_conf.rxmode.mq_mode =
> ETH_MQ_RX_VMDQ_ONLY;
> > -			if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1)
> > -				RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1;
> >  			break;
> The nb_q_per_pool of (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool  would
> be set as 1, that means for VF the number of max queue can be used would
> be only 1.
> 
> And next we start VF using l3fwd:
> l3fwd/build/l3fwd -c 0xc0000 -n 4 -w 0000:05:10.0 -w 0000:05:10.1 -- -p 0x3
> --config '(0,0,18),(1,0,19)' --parse-ptype The number of rx_q equal to 1 and
> there is no problem. But, in l3fwd, the number of tx_q would equal to the
> number of core used in l3fwd, so here number of tx_q equal to 2 but as
> described above, the max number of queue can be used in VF is just only 1.
> So there is problem and l3fwd will check failed.
> 
> 
> So there is two ways to fix this bug, the first is using this patch, don't assign
> RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool as 1 when mq_mode =
> ETH_MQ_RX_VMDQ_ONLY.
> And the second is that the number of tx_q would not assigned as the number
> of cores used in l3fwd.
> 
> Which one do you think is make sense?
> 
> 
> 
> 
> Yanglong
> 
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Wu, Yanglong
> Sent: Thursday, January 11, 2018 6:30 PM
> To: Dai, Wei <wei.dai at intel.com>; dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF
> 
> Yes, this problem is caused by PF passing a wrong number of queue to VF. So
> basically, I removed  model check of "case ETH_MQ_RX_NONE". It seem
> right after I do that but I'am also not very sure the changing is really right. So
> thank to your comments, I will redebug this problem.
> 
> -----Original Message-----
> From: Dai, Wei
> Sent: Thursday, January 11, 2018 5:48 PM
> To: Wu, Yanglong <yanglong.wu at intel.com>; dev at dpdk.org; Yang, Zhiyong
> <zhiyong.yang at intel.com>
> Cc: Wu, Yanglong <yanglong.wu at intel.com>
> Subject: RE: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF
> 
> According to 82599 datasheet, VF working mode is controlled by its host PF.
> If VF is working in RSS or DCB mode, it can use more than 1 queues.
> That also means its host PF should enable RSS or DCB.
> The PF Rx mode can be ETH_MQ_RX_VMDQ_RSS or
> ETH_MQ_RX_VMDQ_DCB.
> So I think that DPDK app with PF cause this issue, not ixgbe PMD itself.
> 
> By the way, you can reply your patch mail to provide more info about how
> You reproduce this issue to easy others' reviewing and testing your patch.
> 
> Thanks
> 
> > -----Original Message-----
> > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Yanglong Wu
> > Sent: Tuesday, January 9, 2018 2:32 PM
> > To: dev at dpdk.org; Yang, Zhiyong <zhiyong.yang at intel.com>
> > Cc: Wu, Yanglong <yanglong.wu at intel.com>
> > Subject: [dpdk-dev] [PATCH v4] net/ixgbe: fix l3fwd start failed on VF
> >
> > VF can't run in multi queue mode, if nb_q_per_pool was set as 1.
> > Nb_q_per_pool is passed through to max_rx_q and max_tx_q in VF.
> > So if nb_q_per_pool is equal to 1, max_rx_q and max_tx_q can't be more
> > than 1 and VF multi queue mode will fail.
> >
> > Fixes: 27b609cbd1c6 (ethdev: move the multi-queue mode check)
> >
> > Signed-off-by: Yanglong Wu <yanglong.wu at intel.com>
Acked-by: Wei Dai <wei.dai at intel.com>
> > ---
> > v1~v3:
> > coding style issue fix
> > ---
> > v4:
> > Rework according to comments
> > ---
> >  drivers/net/ixgbe/ixgbe_ethdev.c | 2 --
> >  1 file changed, 2 deletions(-)
> >
> > diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> > b/drivers/net/ixgbe/ixgbe_ethdev.c
> > index baaeee5d9..6af4f8b2b 100644
> > --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> > +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> > @@ -2240,8 +2240,6 @@ ixgbe_check_mq_mode(struct rte_eth_dev
> *dev)
> >  		case ETH_MQ_RX_NONE:
> >  			/* if nothing mq mode configure, use default scheme */
> >  			dev->data->dev_conf.rxmode.mq_mode =
> ETH_MQ_RX_VMDQ_ONLY;
> > -			if (RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool > 1)
> > -				RTE_ETH_DEV_SRIOV(dev).nb_q_per_pool = 1;
> >  			break;
> >  		default: /* ETH_MQ_RX_DCB, ETH_MQ_RX_DCB_RSS or
> ETH_MQ_TX_DCB*/
> >  			/* SRIOV only works in VMDq enable mode */
> > --
> > 2.11.0