[v4] net/af_xdp: support need wakeup feature

Message ID 20190628100705.45988-1-xiaolong.ye@intel.com (mailing list archive)
State Accepted, archived
Delegated to: Ferruh Yigit
Headers
Series [v4] net/af_xdp: support need wakeup feature |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation fail Compilation issues
ci/intel-Performance-Testing success Performance Testing PASS
ci/mellanox-Performance-Testing success Performance Testing PASS

Commit Message

Xiaolong Ye June 28, 2019, 10:07 a.m. UTC
  This patch enables need_wakeup flag for Tx and fill rings, when this flag
is set by the driver, it means that the userspace application has to
explicitly wake up the kernel Rx or kernel Tx processing by issuing a
syscall. Poll() can wake up both and sendto() or its alternatives will wake
up Tx processing only.

This feature is to provide efficient support for case that application and
driver executing on the same core.

Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
---

v4 changes:

1. document need_wakeup feature in the af_xdp.rst

v3 changes:

1. add introduction in 19.08 release note

v2 changes:

1. remove need_wakeup devarg to make need_wakeup feature enabled
   unconditionally.
2. add conditional compilation directive to avoid breaking build with
   kernel which doesn't support need_wakeup feature yet.

Note:

Original busy poll feature has morphed into need_wakeup flag in
kernel side, the main purpose is the same, that is to support both
application and driver executing on the same core efficiently.

 doc/guides/nics/af_xdp.rst             |  7 +++++
 doc/guides/rel_notes/release_19_08.rst |  2 ++
 drivers/net/af_xdp/rte_eth_af_xdp.c    | 41 +++++++++++++++++++-------
 3 files changed, 39 insertions(+), 11 deletions(-)
  

Comments

Ferruh Yigit June 28, 2019, 5:58 p.m. UTC | #1
On 6/28/2019 11:07 AM, Xiaolong Ye wrote:
> This patch enables need_wakeup flag for Tx and fill rings, when this flag
> is set by the driver, it means that the userspace application has to
> explicitly wake up the kernel Rx or kernel Tx processing by issuing a
> syscall. Poll() can wake up both and sendto() or its alternatives will wake
> up Tx processing only.
> 
> This feature is to provide efficient support for case that application and
> driver executing on the same core.
> 
> Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>

Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

Applied to dpdk-next-net/master, thanks.
  

Patch

diff --git a/doc/guides/nics/af_xdp.rst b/doc/guides/nics/af_xdp.rst
index 18defcda3..c99c4c715 100644
--- a/doc/guides/nics/af_xdp.rst
+++ b/doc/guides/nics/af_xdp.rst
@@ -21,6 +21,12 @@  be added later.
 Note that MTU of AF_XDP PMD is limited due to XDP lacks support for
 fragmentation.
 
+AF_XDP PMD enables need_wakeup flag by default if it is supported. This
+need_wakeup feature is used to support executing application and driver on the
+same core efficiently. This feature not only has a large positive performance
+impact for the one core case, but also does not degrade 2 core performance and
+actually improves it for Tx heavy workloads.
+
 Options
 -------
 
@@ -41,6 +47,7 @@  This is a Linux-specific PMD, thus the following prerequisites apply:
    User can install libbpf via `make install_lib` && `make install_headers` in
    <kernel src tree>/tools/lib/bpf;
 *  A Kernel bound interface to attach to;
+*  For need_wakeup feature, it requires kernel version later than v5.3-rc1;
 
 Set up an af_xdp interface
 -----------------------------
diff --git a/doc/guides/rel_notes/release_19_08.rst b/doc/guides/rel_notes/release_19_08.rst
index 3da266705..1c9efcbe7 100644
--- a/doc/guides/rel_notes/release_19_08.rst
+++ b/doc/guides/rel_notes/release_19_08.rst
@@ -93,6 +93,8 @@  New Features
     high performance
   * Added multi-queue support to allow one af_xdp vdev with multiple netdev
     queues
+  * Enabled need_wakeup feature which can provide efficient support for case
+    that application and driver executing on the same core.
 
 * **Updated telemetry library for global metrics support.**
 
diff --git a/drivers/net/af_xdp/rte_eth_af_xdp.c b/drivers/net/af_xdp/rte_eth_af_xdp.c
index c638d9227..5ce90a760 100644
--- a/drivers/net/af_xdp/rte_eth_af_xdp.c
+++ b/drivers/net/af_xdp/rte_eth_af_xdp.c
@@ -5,6 +5,7 @@ 
 #include <errno.h>
 #include <stdlib.h>
 #include <string.h>
+#include <poll.h>
 #include <netinet/in.h>
 #include <net/if.h>
 #include <sys/socket.h>
@@ -90,6 +91,7 @@  struct pkt_rx_queue {
 	struct rx_stats stats;
 
 	struct pkt_tx_queue *pair;
+	struct pollfd fds[1];
 	int xsk_queue_idx;
 };
 
@@ -206,8 +208,14 @@  eth_af_xdp_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts)
 		return 0;
 
 	rcvd = xsk_ring_cons__peek(rx, nb_pkts, &idx_rx);
-	if (rcvd == 0)
+	if (rcvd == 0) {
+#if defined(XDP_USE_NEED_WAKEUP)
+		if (xsk_ring_prod__needs_wakeup(fq))
+			(void)poll(rxq->fds, 1, 1000);
+#endif
+
 		goto out;
+	}
 
 	if (xsk_prod_nb_free(fq, free_thresh) >= free_thresh)
 		(void)reserve_fill_queue(umem, ETH_AF_XDP_RX_BATCH_SIZE);
@@ -279,16 +287,19 @@  kick_tx(struct pkt_tx_queue *txq)
 {
 	struct xsk_umem_info *umem = txq->pair->umem;
 
-	while (send(xsk_socket__fd(txq->pair->xsk), NULL,
-		      0, MSG_DONTWAIT) < 0) {
-		/* some thing unexpected */
-		if (errno != EBUSY && errno != EAGAIN && errno != EINTR)
-			break;
-
-		/* pull from completion queue to leave more space */
-		if (errno == EAGAIN)
-			pull_umem_cq(umem, ETH_AF_XDP_TX_BATCH_SIZE);
-	}
+#if defined(XDP_USE_NEED_WAKEUP)
+	if (xsk_ring_prod__needs_wakeup(&txq->tx))
+#endif
+		while (send(xsk_socket__fd(txq->pair->xsk), NULL,
+			    0, MSG_DONTWAIT) < 0) {
+			/* some thing unexpected */
+			if (errno != EBUSY && errno != EAGAIN && errno != EINTR)
+				break;
+
+			/* pull from completion queue to leave more space */
+			if (errno == EAGAIN)
+				pull_umem_cq(umem, ETH_AF_XDP_TX_BATCH_SIZE);
+		}
 	pull_umem_cq(umem, ETH_AF_XDP_TX_BATCH_SIZE);
 }
 
@@ -622,6 +633,11 @@  xsk_configure(struct pmd_internals *internals, struct pkt_rx_queue *rxq,
 	cfg.libbpf_flags = 0;
 	cfg.xdp_flags = XDP_FLAGS_UPDATE_IF_NOEXIST;
 	cfg.bind_flags = 0;
+
+#if defined(XDP_USE_NEED_WAKEUP)
+	cfg.bind_flags |= XDP_USE_NEED_WAKEUP;
+#endif
+
 	ret = xsk_socket__create(&rxq->xsk, internals->if_name,
 			rxq->xsk_queue_idx, rxq->umem->umem, &rxq->rx,
 			&txq->tx, &cfg);
@@ -683,6 +699,9 @@  eth_rx_queue_setup(struct rte_eth_dev *dev,
 		goto err;
 	}
 
+	rxq->fds[0].fd = xsk_socket__fd(rxq->xsk);
+	rxq->fds[0].events = POLLIN;
+
 	rxq->umem->pmd_zc = internals->pmd_zc;
 
 	dev->data->rx_queues[rx_queue_id] = rxq;