[dpdk-dev] Why only rx queue "0" can receive network packet by i40e NIC

[Jeff Venable, Sr.]

Hi Helin,

We do not want RSS to include L4 ports in the hash because packet fragments would get routed to queue #0 and would be more difficult to work with.  We are using the model where multiple CPUs are pulling from the NIC queues independently with no shared state, so each 'pipeline' has private fragment reassembly state for the sessions it is managing.

Getting RSS Toeplitz hash to work on { source_ip, dest_ip } tuples only using a symmetric rss-key is important.  This works properly with all other Intel NICs in the DPDK thus far that we have tested until the i40E PMD with the Intel X710-DA4.  The Microsoft RSS specification allows for this.

With the i40E PMD, we have been unsuccessful at enabling this RSS configuration.  From the source code and XL710 controller datasheet, we cannot find any reference to the flags for this RSS mode.  Unless we can achieve feature parity with the other Intel NICs, we don't want to write special case code for this one driver which makes the XL710 controller unusable for us and seems contrary to the intent of the DPDK APIs which are abstracting this behavior.

Do you have any suggestions?

Thanks kindly,

Jeff

-----Original Message-----
From: Zhang, Helin [mailto:helin.zhang at intel.com] 
Sent: Wednesday, July 22, 2015 5:56 PM
To: Jeff Venable, Sr. <jeff at vectranetworks.com>; lhffjzh <lhffjzh at 126.com>; 'Thomas Monjalon' <thomas.monjalon at 6wind.com>
Cc: dev at dpdk.org
Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network packet by i40e NIC



> -----Original Message-----
> From: Jeff Venable, Sr. [mailto:jeff at vectranetworks.com]
> Sent: Wednesday, July 22, 2015 5:47 PM
> To: Zhang, Helin; lhffjzh; 'Thomas Monjalon'
> Cc: dev at dpdk.org
> Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> packet by i40e NIC
> 
> Is the I40E incapable of operating RSS with ETH_RSS_IP (i.e. hashing 
> without L4 ports)?
Why do you think like this? Sorry, I am a bit confused.
ETH_RSS_IP is a super set of all IP based rss types. Please see the rss types listed in rte_ethdev.h.
The supports rss types of each NIC can be queried via 'struct rte_eth_dev_info' of field 'flow_type_rss_offloads'.

Regards,
Helin

> 
> Thanks,
> 
> Jeff
> 
> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Zhang, Helin
> Sent: Saturday, February 28, 2015 6:34 AM
> To: lhffjzh; 'Thomas Monjalon'
> Cc: dev at dpdk.org; maintainers at dpdk.org
> Subject: Re: [dpdk-dev] Why only rx queue "0" can receive network 
> packet by i40e NIC
> 
> Good to know that!
> 
> > -----Original Message-----
> > From: lhffjzh [mailto:lhffjzh at 126.com]
> > Sent: Saturday, February 28, 2015 12:34 PM
> > To: Zhang, Helin; 'Thomas Monjalon'
> > Cc: dev at dpdk.org; maintainers at dpdk.org
> > Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> > packet by i40e NIC
> >
> > Hi Helin,
> >
> > Thanks a lot for your great help, all of rx queue received network 
> > packet after I update rss_hf from "ETH_RSS_IP" to " ETH_RSS_PROTO_MASK ".
> >
> > static struct rte_eth_conf port_conf = {
> >     .rxmode = {
> >         .mq_mode        = ETH_MQ_RX_RSS,
> >         .max_rx_pkt_len = ETHER_MAX_LEN,
> >         .split_hdr_size = 0,
> >         .header_split   = 0, /**< Header Split disabled */
> >         .hw_ip_checksum = 1, /**< IP checksum offload enabled */
> >         .hw_vlan_filter = 0, /**< VLAN filtering disabled */
> >         .jumbo_frame    = 0, /**< Jumbo Frame Support disabled */
> >         .hw_strip_crc   = 0, /**< CRC stripped by hardware */
> >     },
> >     .rx_adv_conf = {
> >         .rss_conf = {
> >             .rss_key = NULL,
> >             .rss_hf = ETH_RSS_PROTO_MASK,
> >         },
> >     },
> >     .txmode = {
> >         .mq_mode = ETH_MQ_TX_NONE,
> >     },
> >     .fdir_conf.mode = RTE_FDIR_MODE_SIGNATURE, };
> >
> >
> > Regards,
> > Haifeng
> >
> > -----Original Message-----
> > From: Zhang, Helin [mailto:helin.zhang at intel.com]
> > Sent: Saturday, February 28, 2015 11:18 AM
> > To: lhffjzh; 'Thomas Monjalon'
> > Cc: dev at dpdk.org; maintainers at dpdk.org
> > Subject: RE: [dpdk-dev] Why only rx queue "0" can receive network 
> > packet by i40e NIC
> >
> > Hi Haifeng
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of lhffjzh
> > > Sent: Saturday, February 28, 2015 9:48 AM
> > > To: 'Thomas Monjalon'
> > > Cc: dev at dpdk.org; maintainers at dpdk.org
> > > Subject: Re: [dpdk-dev] Why only rx queue "0" can receive network 
> > > packet
> > by
> > > i40e NIC
> > >
> > > Hi Thomas,
> > >
> > > Thanks very much for your reminder, you give me many help in this 
> > > mail
> > list.
> > >
> > > The issue with detailed information just as below. but I don't 
> > > know who is
> > the
> > > dpdk i40e maintainers? is maintainers at dpdk.org?
> > >
> > > Hardware list:
> > >     2 i40e 40G NICs
> > >     Xeon E5-2670 v2(10 cores)
> > >     32G memory
> > >
> > > I loopback 2 i40e NICs by QSFP cable, one NIC send UDP network 
> > > packet by DPDK, and another for receiving. I bind 4 processor's 
> > > logical cores with 4
> > rx
> > > queue "0,1,2,3" on receiving NIC, when I start to send packet, 
> > > only rx
> > queue
> > > "0"
> > > can receive
> > > the UDP packet, the others queue always receive nothing. but it is 
> > > work
> > well on
> > > ixgbe 10G NICs, I can receive network packet from all rx queues.
> > > does
> > anyone
> > > kindly know why?
> > Could you help to list the DPDK version you are using now?
> > Two possible reasons:
> > 1. UDP rss is not enabled on your board correctly.
> > 	I40e has different rss flags from ixgbe, so I am wondering if you 
> > use it correctly.
> > 	In addition, this will be unified from 2.0. So I care about the DPDK version.
> > 2. The UDP stream is occasionally hit the hash key of queue 0.
> > 	You'd better to try to send your UDP stream with random 5-tuples, 
> > to get the
> > 	hash value hit different queues randomly.
> >
> > Regards,
> > Helin
> >
> > >
> > >
> > > Regards,
> > > Haifeng
> > >
> > > -----Original Message-----
> > > From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
> > > Sent: Friday, February 27, 2015 6:55 PM
> > > To: lhffjzh
> > > Cc: dev at dpdk.org
> > > Subject: Re: Why only rx queue "0" can receive network packet by 
> > > i40e NIC
> > >
> > > 2015-02-27 16:47, lhffjzh:
> > > > Hi All,
> > > >
> > > > We use 4 cores loop 4 rx queues on one i40e port, but only rx queue "0"
> > > can
> > > > receive network packet, do anyone kindly know why? BTW, all of 
> > > > network packet has same destination ip address but has more than
> > > > 200 different source ip address.
> > >
> > > It's possible that you don't have any answer for 2 reasons:
> > > - you replied in a thread dedicated to Cisco enic questions
> > > - you didn't describe your usage enough to understand your problem
> > >
> > > I suggest to use the button "new email" instead of "reply all" to 
> > > start a new question with enough details.
> > >
> > > Did you noticed you put some Cisco guys in CC instead of putting 
> > > the Intel responsible for i40e (see MAINTAINERS file)?
> > >
> >