[dpdk-dev] [PATCH v5 10/10] doc: add application usage guide for l2fwd-event
Jerin Jacob
jerinjacobk at gmail.com
Fri Oct 11 16:11:23 CEST 2019
On Thu, Oct 3, 2019 at 2:29 AM <pbhagavatula at marvell.com> wrote:
>
> From: Sunil Kumar Kori <skori at marvell.com>
>
> Add documentation for l2fwd-event example.
> Update release notes.
>
> Signed-off-by: Sunil Kumar Kori <skori at marvell.com>
# Please fix the typos through with the following command
aspell --lang=en_US --check doc/guides/sample_app_ug/l2_forward_event.rst
# Remove the following warning.
$ make doc-guides-html
doc/guides/sample_app_ug/l2_forward_real_virtual.rst:39: WARNING:
duplicate label figure_l2_fwd_benchmark_setup,
> ---
> MAINTAINERS | 1 +
> doc/guides/rel_notes/release_19_11.rst | 6 +
> doc/guides/sample_app_ug/index.rst | 1 +
> doc/guides/sample_app_ug/intro.rst | 5 +
> doc/guides/sample_app_ug/l2_forward_event.rst | 755 ++++++++++++++++++
> 5 files changed, 768 insertions(+)
> create mode 100644 doc/guides/sample_app_ug/l2_forward_event.rst
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 292ac10c3..94a49b812 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1461,6 +1461,7 @@ F: examples/l2fwd-cat/
> M: Sunil Kumar Kori <skori at marvell.com>
> M: Pavan Nikhilesh <pbhagavatula at marvell.com>
> F: examples/l2fwd-event/
> +F: doc/guides/sample_app_ug/l2_forward_event.rst
> T: git://dpdk.org/next/dpdk-next-eventdev
>
> F: examples/l3fwd/
> diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
> index 27cfbd9e3..071593e4d 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -56,6 +56,12 @@ New Features
> Also, make sure to start the actual text at the margin.
> =========================================================
>
> +* **Added new example l2fwd-event**
Added new example l2fwd-event application
> +
> + Added an example application `l2fwd-event` that adds event device support to
> + traditional l2fwd example. The default poll mode is also preserved for
> + readability.
Please change the last sentence to the following or something similar
"It demonstrates usage of poll and event mode IO mechanism under a
single application"
> +
>
> Removed Items
> -------------
> diff --git a/doc/guides/sample_app_ug/index.rst b/doc/guides/sample_app_ug/index.rst
> index f23f8f59e..41388231a 100644
> --- a/doc/guides/sample_app_ug/index.rst
> +++ b/doc/guides/sample_app_ug/index.rst
> @@ -26,6 +26,7 @@ Sample Applications User Guides
> l2_forward_crypto
> l2_forward_job_stats
> l2_forward_real_virtual
> + l2_forward_event
> l2_forward_cat
> l3_forward
> l3_forward_power_man
> diff --git a/doc/guides/sample_app_ug/intro.rst b/doc/guides/sample_app_ug/intro.rst
> index 90704194a..84591c0a1 100644
> --- a/doc/guides/sample_app_ug/intro.rst
> +++ b/doc/guides/sample_app_ug/intro.rst
> @@ -87,6 +87,11 @@ examples are highlighted below.
> forwarding, or ``l2fwd`` application does forwarding based on Ethernet MAC
> addresses like a simple switch.
>
> +* :doc:`Network Layer 2 forwarding<l2_forward_event>`: The Network Layer 2
> + forwarding, or ``l2fwd-event`` application does forwarding based on Ethernet MAC
> + addresses like a simple switch. It demonstrate usage of poll and event mode Rx/Tx
> + mechanism.
Please change the last sentence to the following or something similar
"It demonstrates usage of poll and event mode IO mechanism under a
single application"
> +
> * :doc:`Network Layer 3 forwarding<l3_forward>`: The Network Layer3
> forwarding, or ``l3fwd`` application does forwarding based on Internet
> Protocol, IPv4 or IPv6 like a simple router.
> diff --git a/doc/guides/sample_app_ug/l2_forward_event.rst b/doc/guides/sample_app_ug/l2_forward_event.rst
> new file mode 100644
> index 000000000..250d16887
> --- /dev/null
> +++ b/doc/guides/sample_app_ug/l2_forward_event.rst
> @@ -0,0 +1,755 @@
> +.. SPDX-License-Identifier: BSD-3-Clause
> + Copyright(c) 2010-2014 Intel Corporation.
> +
> +.. _l2_fwd_event_app:
> +
> +L2 Forwarding Eventdev Sample Application
> +=========================================
> +
> +The L2 Forwarding eventdev sample application is a simple example of packet
> +processing using the Data Plane Development Kit (DPDK) to demonstrate usage of
> +poll and event mode packet I/O mechanism.
> +
> +Overview
> +--------
> +
> +The L2 Forwarding eventdev sample application, performs L2 forwarding for each
> +packet that is received on an RX_PORT. The destination port is the adjacent port
> +from the enabled portmask, that is, if the first four ports are enabled (portmask=0x0f),
> +ports 1 and 2 forward into each other, and ports 3 and 4 forward into each other.
> +Also, if MAC addresses updating is enabled, the MAC addresses are affected as follows:
> +
> +* The source MAC address is replaced by the TX_PORT MAC address
> +
> +* The destination MAC address is replaced by 02:00:00:00:00:TX_PORT_ID
> +
> +Appliation receives packets from RX_PORT using below mentioned methods:
> +
> +* Poll mode
> +
> +* Eventdev mode (default)
> +
> +This application can be used to benchmark performance using a traffic-generator,
> +as shown in the :numref:`figure_l2_fwd_benchmark_setup`.
> +
> +.. _figure_l2_fwd_benchmark_setup:
> +
> +.. figure:: img/l2_fwd_benchmark_setup.*
Looks like there is a typo in the original l2fwd application image.
The diagram shows NUT instead of DUT. Please send a separate patch.
> +
> + Performance Benchmark Setup (Basic Environment)
> +
> +Compiling the Application
> +-------------------------
> +
> +To compile the sample application see :doc:`compiling`.
> +
> +The application is located in the ``l2fwd-event`` sub-directory.
> +
> +Running the Application
> +-----------------------
> +
> +The application requires a number of command line options:
> +
> +.. code-block:: console
> +
> + ./build/l2fwd-event [EAL options] -- -p PORTMASK [-q NQ] --[no-]mac-updating --mode=MODE --eventq-sync=SYNC_MODE
> +
> +where,
> +
> +* p PORTMASK: A hexadecimal bitmask of the ports to configure
> +
> +* q NQ: A number of queues (=ports) per lcore (default is 1)
> +
> +* --[no-]mac-updating: Enable or disable MAC addresses updating (enabled by default).
> +
> +* --mode=MODE: Packet transfer mode for I/O, poll or eventdev. Eventdev by default.
> +
> +* --eventq-sync=SYNC_MODE: Event queue synchronization method, Ordered or Atomic. Atomic by default.
> +
> +Sample usage commands are given below to run the application into different mode:
> +
> +Poll mode on linux environment with 4 lcores, 16 ports and 8 RX queues per lcore
Nothing specific to Linux here, so please remove Linux environment
> +Driver Initialization
> +~~~~~~~~~~~~~~~~~~~~~
> +
> +The main part of the code in the main() function relates to the initialization
> +of the driver. To fully understand this code, it is recommended to study the
> +chapters that related to the Poll Mode and Event mode Driver in the
> +*DPDK Programmer's Guide* - Rel 1.4 EAR and the *DPDK API Reference*.
> +
> +.. code-block:: c
> +
> + if (rte_pci_probe() < 0)
> + rte_exit(EXIT_FAILURE, "Cannot probe PCI\n");
> +
> + /* reset l2fwd_dst_ports */
> +
> + for (portid = 0; portid < RTE_MAX_ETHPORTS; portid++)
> + l2fwd_dst_ports[portid] = 0;
> +
> + last_port = 0;
> +
> + /*
> + * Each logical core is assigned a dedicated TX queue on each port.
> + */
> +
> + RTE_ETH_FOREACH_DEV(portid) {
> + /* skip ports that are not enabled */
> +
> + if ((l2fwd_enabled_port_mask & (1 << portid)) == 0)
> + continue;
> +
> + if (nb_ports_in_mask % 2) {
> + l2fwd_dst_ports[portid] = last_port;
> + l2fwd_dst_ports[last_port] = portid;
> + }
> + else
> + last_port = portid;
> +
> + nb_ports_in_mask++;
> +
> + rte_eth_dev_info_get((uint8_t) portid, &dev_info);
> + }
> +
> +Observe that:
> +
> +* rte_igb_pmd_init_all() simultaneously registers the driver as a PCI driver
Looks like rte_igb_pmd_init_all() from legacy l2fwd doc, Please fix in
both docs.
The original l2fwd doc fix send it as a separate patch.
> +.. code-block:: c
> +
> + /* Start event device service */
> + ret = rte_event_dev_service_id_get(eventdev_rsrc.event_d_id,
> + &service_id);
> + if (ret != -ESRCH && ret != 0)
> + rte_exit(EXIT_FAILURE, "Error in starting eventdev");
> +
> + rte_service_runstate_set(service_id, 1);
> + rte_service_set_runstate_mapped_check(service_id, 0);
> + eventdev_rsrc.service_id = service_id;
eventdev_rsrc removed recently in code review, Please sync code with
sync in all code sections.
> +.. note::
> +
> + In the following code, one line for getting the output port requires some
> + explanation.
Do we need this note: ?
> +
> +During the initialization process, a static array of destination ports
> +(l2fwd_dst_ports[]) is filled such that for each source port, a destination port
> +is assigned that is either the next or previous enabled port from the portmask.
> +If number of ports are odd in portmask then packet from last port will be
> +forwarded to first port i.e. if portmask=0x07, then forwarding will take place
> +like p0--->p1, p1--->p2, p2--->p0.
> +
> +Also to optimize enqueue opeartion, l2fwd_simple_forward() stores incoming mbus
> +upto MAX_PKT_BURST. Once it reaches upto limit, all packets are transmitted to
> +destination ports.
> +
> +.. code-block:: c
> +
> + static void
> + l2fwd_simple_forward(struct rte_mbuf *m, uint32_t portid)
> + {
> + uint32_t dst_port;
> + int32_t sent;
> + struct rte_eth_dev_tx_buffer *buffer;
> +
> + dst_port = l2fwd_dst_ports[portid];
> +
> + if (mac_updating)
> + l2fwd_mac_updating(m, dst_port);
> +
> + buffer = tx_buffer[dst_port];
> + sent = rte_eth_tx_buffer(dst_port, 0, buffer, m);
> + if (sent)
> + port_statistics[dst_port].tx += sent;
> + }
> +
> +For this test application, the processing is exactly the same for all packets
> +arriving on the same RX port. Therefore, it would have been possible to call
> +the rte_eth_tx_buffer() function directly from the main loop to send all the
> +received packets on the same TX port, using the burst-oriented send function,
> +which is more efficient.
> +
> +However, in real-life applications (such as, L3 routing),
> +packet N is not necessarily forwarded on the same port as packet N-1.
> +The application is implemented to illustrate that, so the same approach can be
> +reused in a more complex application.
> +
> +To ensure that no packets remain in the tables, each lcore does a draining of TX
> +queue in its main loop. This technique introduces some latency when there are
> +not many packets to send, however it improves performance:
> +
> +.. code-block:: c
> +
> + cur_tsc = rte_rdtsc();
> +
> + /*
> + * TX burst queue drain
> + */
> + diff_tsc = cur_tsc - prev_tsc;
> + if (unlikely(diff_tsc > drain_tsc)) {
> + for (i = 0; i < qconf->n_rx_port; i++) {
> + portid = l2fwd_dst_ports[qconf->rx_port_list[i]];
> + buffer = tx_buffer[portid];
> + sent = rte_eth_tx_buffer_flush(portid, 0,
> + buffer);
> + if (sent)
> + port_statistics[portid].tx += sent;
> + }
> +
> + /* if timer is enabled */
> + if (timer_period > 0) {
> + /* advance the timer */
> + timer_tsc += diff_tsc;
> +
> + /* if timer has reached its timeout */
> + if (unlikely(timer_tsc >= timer_period)) {
> + /* do this only on master core */
> + if (lcore_id == rte_get_master_lcore()) {
> + print_stats();
> + /* reset the timer */
> + timer_tsc = 0;
> + }
> + }
> + }
> +
> + prev_tsc = cur_tsc;
> + }
> +
> +In the **l2fwd_main_loop_eventdev()** function, the main task is to read ingress
> +packets from the event ports. This is done using the following code:
> +
> +.. code-block:: c
> +
> + /* Read packet from eventdev */
> + nb_rx = rte_event_dequeue_burst(event_d_id, event_p_id,
> + events, deq_len, 0);
> + if (nb_rx == 0) {
> + rte_pause();
> + continue;
> + }
> +
> + for (i = 0; i < nb_rx; i++) {
> + mbuf[i] = events[i].mbuf;
> + rte_prefetch0(rte_pktmbuf_mtod(mbuf[i], void *));
> + }
> +
> +
> +Before reading packets, deq_len is fetched to ensure correct allowed deq length
> +by the eventdev.
> +The rte_event_dequeue_burst() function writes the mbuf pointers in a local table
> +and returns the number of available mbufs in the table.
> +
> +Then, each mbuf in the table is processed by the l2fwd_eventdev_forward()
> +function. The processing is very simple: process the TX port from the RX port,
> +then replace the source and destination MAC addresses if MAC addresses updating
> +is enabled.
> +
> +.. note::
> +
> + In the following code, one line for getting the output port requires some
> + explanation.
Do we need this note: ?
> +
> +During the initialization process, a static array of destination ports
> +(l2fwd_dst_ports[]) is filled such that for each source port, a destination port
> +is assigned that is either the next or previous enabled port from the portmask.
> +If number of ports are odd in portmask then packet from last port will be
> +forwarded to first port i.e. if portmask=0x07, then forwarding will take place
> +like p0--->p1, p1--->p2, p2--->p0.
> +
> +l2fwd_eventdev_forward() does not stores incoming mbufs. Packet will forwarded
> +be to destination ports via Tx adapter or generic event dev enqueue API
> +depending H/W or S/W scheduler is used.
> +
> +.. code-block:: c
> +
> + static inline void
> + l2fwd_eventdev_forward(struct rte_mbuf *m[], uint32_t portid,
> + uint16_t nb_rx, uint16_t event_p_id)
> + {
> + uint32_t dst_port, i;
> +
> + dst_port = l2fwd_dst_ports[portid];
> +
> + for (i = 0; i < nb_rx; i++) {
> + if (mac_updating)
> + l2fwd_mac_updating(m[i], dst_port);
> +
> + m[i]->port = dst_port;
> + }
> +
> + if (timer_period > 0) {
> + rte_spinlock_lock(&port_stats_lock);
> + port_statistics[dst_port].tx += nb_rx;
> + rte_spinlock_unlock(&port_stats_lock);
> + }
> + /* Registered callback is invoked for Tx */
> + eventdev_rsrc.send_burst_eventdev(m, nb_rx, event_p_id);
> + }
> --
> 2.17.1
>
More information about the dev
mailing list