[dpdk-dev] App acting strangely when Vector rx / FDIR is on

[Dor Green]

"Fixed" the problem.

I upgraded to DPDK 2.0, which did not allow vector rx to until I have
changed my RX hardware queue size to a power of two.
After I did that (and changed the rx_free_thresh accordingly), it simply worked.

On Sun, Apr 5, 2015 at 11:45 AM, Dor Green <dorgreen1 at gmail.com> wrote:
> I have an app which processes packets, and for a while I had FDIR in
> signature mode. Since I no longer needed filtering, I turned it off
> (which also turned on vector rx) which caused some strange behaviour.
>
> I changed the app so that it simply prints the length of the packet
> for each incoming one, then sent 10,000 packets at a very slow speed
> (10Mbps) with only one core.
> Most noticable was that even though the NIC stats showed the same
> packets received (rte_eth_stats.ipackets), only 9111 packet lengths
> were printed when vector/fdir was off (and the entire 10,000 were
> printed when it was on).
> The data lengths themselves were also different, but I expected some
> reordering. When looking at a set of all the lengths, there were many
> lengths that only appeared when fdir was on, and only one that
> appeared when it was on but not when it was off.
>
> Can anyone think of any explanation for this phenomenon? Any checks I could do?
>
> Using a 10G 2P X520 adapter. These are the startup logs (when fdir is
> off. When it is on, the "Vector rx enabled" line is removed):
>
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 1 on socket 0
> EAL: Detected lcore 2 as core 2 on socket 0
> EAL: Detected lcore 3 as core 3 on socket 0
> EAL: Detected lcore 4 as core 4 on socket 0
> EAL: Detected lcore 5 as core 5 on socket 0
> EAL: Detected lcore 6 as core 0 on socket 0
> EAL: Detected lcore 7 as core 1 on socket 0
> EAL: Detected lcore 8 as core 2 on socket 0
> EAL: Detected lcore 9 as core 3 on socket 0
> EAL: Detected lcore 10 as core 4 on socket 0
> EAL: Detected lcore 11 as core 5 on socket 0
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 12 lcore(s)
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x400000000 bytes
> EAL: Virtual area found at 0x7f5700000000 (size = 0x400000000)
> EAL: Requesting 16 pages of size 1024MB from socket 0
> EAL: TSC frequency is ~2094901 KHz
> EAL: Master core 0 is ready (tid=a40368c0)
> EAL: Core 7 is ready (tid=10033700)
> EAL: Core 6 is ready (tid=10834700)
> EAL: Core 5 is ready (tid=11035700)
> EAL: Core 4 is ready (tid=11836700)
> EAL: Core 3 is ready (tid=12037700)
> EAL: Core 2 is ready (tid=12838700)
> EAL: Core 1 is ready (tid=13039700)
> EAL: PCI device 0000:04:00.0 on NUMA socket -1
> EAL:   probe driver: 8086:154d rte_ixgbe_pmd
> EAL:   0000:04:00.0 not managed by UIO driver, skipping
> EAL: PCI device 0000:04:00.1 on NUMA socket 0
> EAL:   probe driver: 8086:154d rte_ixgbe_pmd
> EAL:   PCI memory mapped at 0x7f5ba3e84000
> EAL:   PCI memory mapped at 0x7f5ba4045000
> PMD: eth_ixgbe_dev_init(): MAC: 2, PHY: 11, SFP+: 4
> PMD: eth_ixgbe_dev_init(): port 0 vendorID=0x8086 deviceID=0x154d
> MAIN: Finished DPDK memory setup
> PMD: ixgbe_dev_rx_queue_setup(): sw_ring=0x7f5700af0e40
> hw_ring=0x7f59555fae00 dma_addr=0xe155fae00
> PMD: ixgbe_dev_rx_queue_setup(): Rx Burst Bulk Alloc Preconditions are
> satisfied. Rx Burst Bulk Alloc function will be used on port=0,
> queue=0.
> PMD: ixgbe_dev_rx_queue_setup(): Vector rx enabled, please make sure
> RX burst size no less than 32.
> PMD: ixgbe_dev_tx_queue_setup(): sw_ring=0x7f5700af0900
> hw_ring=0x7f595560ae80 dma_addr=0xe1560ae80
> PMD: set_tx_function(): Using full-featured tx code path
> PMD: set_tx_function():  - txq_flags = 0 [IXGBE_SIMPLE_FLAGS=f01]
> PMD: set_tx_function():  - tx_rs_thresh = 32 [RTE_PMD_IXGBE_TX_MAX_BURST=32]
>
>
> My (shortened) configuration:
>
> static struct rte_eth_conf const ethconf = {
>     .link_speed = 0,
>     .link_duplex = 0,
>     .rxmode = {
>         .mq_mode = ETH_MQ_RX_RSS,
>         .max_rx_pkt_len = 2000,
>         .split_hdr_size = 0,
>         .header_split = 0,
>         .hw_ip_checksum = 0,
>         .hw_vlan_filter = 0,
>         .jumbo_frame = 1,
>         .hw_strip_crc = 0,   /**< CRC stripped by hardware */
>     },
>
>     .rx_adv_conf = {
>         .rss_conf = {
>             .rss_key = NULL,
>             .rss_hf = ETH_RSS_IPV4 | ETH_RSS_IPV6,
>         }
>     },
>
>     .fdir_conf = {
>         .mode = RTE_FDIR_MODE_NONE, // Or RTE_FDIR_MODE_SIGNATURE,
>     },
> };
>
> static struct rte_eth_rxconf const rxconf = {
>     .rx_thresh = {
>         .pthresh = 8,
>         .hthresh = 8,
>         .wthresh = 0,
>     },
>
>     .rx_free_thresh = 300,
>     .rx_drop_en = 0,
> };
>
> Thanks.