[dpdk-dev] Surprisingly high TCP ACK packets drop counter

Wang, Shawn xingbow at amazon.com
Fri Nov 1 15:54:20 CET 2013


Hi:

We had the same problem before. It turned out that RSC (receive side
coalescing) is enabled by default in DPDK. So we write this naïve patch to
disable it. This patch is based on DPDK 1.3. Not sure 1.5 has changed it
or not.
After this patch, ACK rate should go back to 14.5Mpps. For details, you
can refer to Intel® 82599 10 GbE Controller Datasheet. (7.11 Receive Side
Coalescing).

From: xingbow <xingbow at amazon.com>
Date: Wed, 21 Aug 2013 11:35:23 -0700
Subject: [PATCH] Disable RSC in ixgbe_dev_rx_init function in file

 ixgbe_rxtx.c
 
---

 DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h | 2 +-
 DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c       | 7 +++++++
 2 files changed, 8 insertions(+), 1 deletion(-)
 
diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h
b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h
index 7fffd60..f03046f 100644

--- a/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h

+++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe/ixgbe_type.h

@@ -1930,7 +1930,7 @@ enum {

 #define IXGBE_RFCTL_ISCSI_DIS		0x00000001
 #define IXGBE_RFCTL_ISCSI_DWC_MASK	0x0000003E
 #define IXGBE_RFCTL_ISCSI_DWC_SHIFT	1
-#define IXGBE_RFCTL_RSC_DIS		0x00000010

+#define IXGBE_RFCTL_RSC_DIS		0x00000020

 #define IXGBE_RFCTL_NFSW_DIS		0x00000040
 #define IXGBE_RFCTL_NFSR_DIS		0x00000080
 #define IXGBE_RFCTL_NFS_VER_MASK	0x00000300
diff --git a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c
index 07830b7..ba6e05d 100755

--- a/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c

+++ b/DPDK/lib/librte_pmd_ixgbe/ixgbe_rxtx.c

@@ -3007,6 +3007,7 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev)

 	uint64_t bus_addr;
 	uint32_t rxctrl;
 	uint32_t fctrl;
+	uint32_t rfctl;

 	uint32_t hlreg0;
 	uint32_t maxfrs;
 	uint32_t srrctl;
@@ -3033,6 +3034,12 @@ ixgbe_dev_rx_init(struct rte_eth_dev *dev)

 	fctrl |= IXGBE_FCTRL_PMCF;
 	IXGBE_WRITE_REG(hw, IXGBE_FCTRL, fctrl);
 
+	/* Disable RSC */
+	RTE_LOG(INFO, PMD, "Disable RSC\n");
+	rfctl = IXGBE_READ_REG(hw, IXGBE_RFCTL);
+	rfctl |= IXGBE_RFCTL_RSC_DIS;
+	IXGBE_WRITE_REG(hw, IXGBE_RFCTL, rfctl);
+

 	/*
 	 * Configure CRC stripping, if any.
 	 */
-- 


Thanks.
Wang, Xingbo




On 11/1/13 6:43 AM, "Alexander Belyakov" <abelyako at gmail.com> wrote:

>Hello,
>
>we have simple test application on top of DPDK which sole purpose is to
>forward as much packets as possible. Generally we easily achieve 14.5Mpps
>with two 82599EB (one as input and one as output). The only suprising
>exception is forwarding pure TCP ACK flood when performace always drops to
>approximately 7Mpps.
>
>For simplicity consider two different types of traffic:
>1) TCP SYN flood is forwarded at 14.5Mpps rate,
>2) pure TCP ACK flood is forwarded only at 7Mpps rate.
>
>Both SYN and ACK packets have exactly the same length.
>
>It is worth to mention, this forwarding application looks at Ethernet and
>IP headers, but never deals with L4 headers.
>
>We tracked down issue to RX circuit. To be specific, there are 4 RX queues
>initialized on input port and rte_eth_stats_get() shows uniform packet
>distribution (q_ipackets) among them, while q_errors remain zero for all
>queues. The only drop counter quickly increasing in the case of pure ACK
>flood is ierrors, while rx_nombuf remains zero.
>
>We tried different kinds of traffic generators, but always got the same
>result: 7Mpps (instead of expected 14Mpps) for TCP packets with ACK flag
>bit set while all other flag bits dropped. Source IPs and ports are
>selected randomly.
>
>Please let us know if anyone is aware of such strange behavior and where
>should we look at to narrow down the problem.
>
>Thanks in advance,
>Alexander Belyakov



More information about the dev mailing list