[dpdk-dev] [PATCH v7 0/6] Support TCP/IPv4, VxLAN, and GRE GSO in DPDK

Mark Kavanagh mark.b.kavanagh at intel.com
Thu Oct 5 13:02:05 CEST 2017

Generic Segmentation Offload (GSO) is a SW technique to split large
packets into small ones. Akin to TSO, GSO enables applications to
operate on large packets, thus reducing per-packet processing overhead.

To enable more flexibility to applications, DPDK GSO is implemented
as a standalone library. Applications explicitly use the GSO library
to segment packets. This patch adds GSO support to DPDK for specific
packet types: specifically, TCP/IPv4, VxLAN, and GRE.

The first patch introduces the GSO API framework. The second patch
adds GSO support for TCP/IPv4 packets (containing an optional VLAN
tag). The third patch adds GSO support for VxLAN packets that contain
outer IPv4, and inner TCP/IPv4 headers (plus optional inner and/or 
outer VLAN tags). The fourth patch adds GSO support for GRE packets
that contain outer IPv4, and inner TCP/IPv4 headers (with optional 
outer VLAN tag). The fifth patch in the series enables TCP/IPv4, VxLAN,
and GRE GSO in testpmd's checksum forwarding engine. The final patch
in the series adds GSO documentation to the programmer's guide.

Performance Testing
The performance of TCP/IPv4 GSO on a 10Gbps link is demonstrated using
iperf. Setup for the test is described as follows:

a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same
   machine, together physically.
b. Launch testpmd with P0 and a vhost-user port, and use csum
   forwarding engine with "retry".
c. Select IP and TCP HW checksum calculation for P0; select TCP HW
   checksum calculation for vhost-user port.
d. Launch a VM with csum and tso offloading enabled.
e. Run iperf-client on virtio-net port in the VM to send TCP packets.
   With enabling csum and tso, the VM can send large TCP/IPv4 packets
   (mss is up to 64KB).
f. P1 is assigned to linux kernel and enabled kernel GRO. Run
   iperf-server on P1.

We conduct three iperf tests:

test-1: enable GSO for P0 in testpmd, and set max GSO segment length
    to 1518B. Run two iperf-client in the VM.
test-2: enable TSO for P0 in testpmd, and set TSO segsz to 1518B. Run
    two iperf-client in the VM.
test-3: disable GSO and TSO in testpmd. Run two iperf-client in the VM.

Throughput of the above three tests:

test-1: 9.4Gbps
test-2: 9.5Gbps
test-3: 3Mbps

Functional Testing
Unlike TCP packets, VMs can't send large VxLAN or GRE packets. The max
length of tunneled packets from VMs is 1514B. So current experiment
method can't be used to measure VxLAN and GRE GSO performance, but simply
test the functionality via setting small GSO segment length (e.g. 500B).

To test VxLAN GSO functionality, we use the following setup:

a. Connect 2 x 10Gbps physical ports (P0, P1), which are in the same
   machine, together physically.
b. Launch testpmd with P0 and a vhost-user port, and use csum forwarding
   engine with "retry".
c. Testpmd commands:
    - csum parse_tunnel on "P0"
    - csum parse_tunnel on "vhost-user port"
    - csum set outer-ip hw "P0"
    - csum set ip hw "P0"
    - csum set tcp hw "P0"
    - csum set tcp hw "vhost-user port"
    - set port "P0" gso on
    - set gso segsz 500
d. Launch a VM with csum and tso offloading enabled.
e. Create a vxlan port for the virtio-net port in the VM. Run iperf-client
   on the VxLAN port, so TCP packets are VxLAN encapsulated. However, the
   max packet length is 1514B.
f. P1 is assigned to linux kernel and kernel GRO is disabled. Similarly,
   create a VxLAN port for P1, and run iperf-server on the VxLAN port.

In testpmd, we can see the length of all packets sent from P0 is smaller
than or equal to 500B. Additionally, the packets arriving in P1 is
encapsulated and is smaller than or equal to 500B.

The same process may be used to test GRE functionality, with the exception that
the tunnel type created for both the guest's virtio-net, and the host's kernel
interfaces is GRE:
   `ip tunnel add <gre tunnel> mode gre remote <remote IP> local <local_ip>`

As in the VxLAN testcase, the length of packets sent from P0, and received on
P1, is less than 500B.

Change log
- add RTE_GSO_SEG_SIZE_MIN macro; use this to validate gso_ctx.gso_segsz.
- rename 'ipid_flag' member of gso_ctx to 'flag'.
- remove mention of VLAN tags in supported packet types.
- don't clear PKT_TX_TCP_SEG flag if GSO fails.
- take all packet overhead into account when checking for empty packet.
- ensure that only enabled GSO types are enacted upon (i.e. no fall-through to
  TCP/IPv4 case from tunneled case).
- validate user-supplied gso segsz arg against RTE_GSO_SEG_SIZE_MIN in testpmd.
- simplify error-checking/handling for GSO failure case in testpmd csum engine.
- use 0 instead of !RTE_GSO_IPID_FIXED in testpmd.

- rebase to HEAD of master (i5dce9fcA)
- remove 'l3_offset' parameter from 'update_ipv4_tcp_headers'

- add GSO section to the programmer's guide.
- use MF or (previously 'and') offset to check if a packet is IP
- move 'update_header' helper functions to gso_common.h.
- move txp/ipv4 'update_header' function to gso_tcp4.c.
- move tunnel 'update_header' function to gso_tunnel_tcp4.c.
- add offset parameter to 'update_header' functions.
- combine GRE and VxLAN tunnel header update functions into a single
- correct typos and errors in comments/commit messages.

- use ol_flags instead of packet_type to decide which segmentation
  function to use.
- use MF and offset to check if a packet is IP fragmented, instead of
  using DF.
- remove ETHER_CRC_LEN from gso segment payload length calculation.
- refactor internal header update and other functions.
- add some of GSO documents.
- set the default GSO length to 1514 and fill PKT_TX_TCP_SEG for the
  packets sent from GSO-enabled ports in testpmd.
- support all IPv4 header flags, including RTE_PTYPE_(INNER_)L3_IPV4,
- fill mbuf->packet_type instead of using rte_net_get_ptype() in
  csumonly.c, since rte_net_get_ptype() doesn't support vxlan.
- store the input packet into pkts_out inside gso_tcp4_segment() and
  gso_tunnel_tcp4_segment() instead of rte_gso_segment(), when no GSO
  is performed.
- add missing incldues.
- optimize file names, function names and function description.
- fix one bug in testpmd.
- merge data segments whose data_len is less than mss into a large data
  segment in gso_do_segment().
- use mbuf->packet_type/l2_len/l3_len etc. instead of parsing the packet
  header in rte_gso_segment().
- provide IP id macros for applications to select fixed or incremental IP

Jiayu Hu (3):
  gso: add Generic Segmentation Offload API framework
  gso: add TCP/IPv4 GSO support
  app/testpmd: enable TCP/IPv4, VxLAN and GRE GSO

Mark Kavanagh (3):
  gso: add VxLAN GSO support
  gso: add GRE GSO support
  doc: add GSO programmer's guide

 MAINTAINERS                                        |   6 +
 app/test-pmd/cmdline.c                             | 179 ++++++++
 app/test-pmd/config.c                              |  24 ++
 app/test-pmd/csumonly.c                            |  43 +-
 app/test-pmd/testpmd.c                             |  13 +
 app/test-pmd/testpmd.h                             |  10 +
 config/common_base                                 |   5 +
 doc/api/doxy-api-index.md                          |   1 +
 doc/api/doxy-api.conf                              |   1 +
 .../generic_segmentation_offload_lib.rst           | 256 +++++++++++
 .../prog_guide/img/gso-output-segment-format.svg   | 313 ++++++++++++++
 doc/guides/prog_guide/img/gso-three-seg-mbuf.svg   | 477 +++++++++++++++++++++
 doc/guides/prog_guide/index.rst                    |   1 +
 doc/guides/rel_notes/release_17_11.rst             |  17 +
 doc/guides/testpmd_app_ug/testpmd_funcs.rst        |  46 ++
 lib/Makefile                                       |   2 +
 lib/librte_eal/common/include/rte_log.h            |   1 +
 lib/librte_gso/Makefile                            |  52 +++
 lib/librte_gso/gso_common.c                        | 153 +++++++
 lib/librte_gso/gso_common.h                        | 171 ++++++++
 lib/librte_gso/gso_tcp4.c                          | 104 +++++
 lib/librte_gso/gso_tcp4.h                          |  74 ++++
 lib/librte_gso/gso_tunnel_tcp4.c                   | 126 ++++++
 lib/librte_gso/gso_tunnel_tcp4.h                   |  75 ++++
 lib/librte_gso/rte_gso.c                           | 111 +++++
 lib/librte_gso/rte_gso.h                           | 148 +++++++
 lib/librte_gso/rte_gso_version.map                 |   7 +
 mk/rte.app.mk                                      |   1 +
 28 files changed, 2413 insertions(+), 4 deletions(-)
 create mode 100644 doc/guides/prog_guide/generic_segmentation_offload_lib.rst
 create mode 100644 doc/guides/prog_guide/img/gso-output-segment-format.svg
 create mode 100644 doc/guides/prog_guide/img/gso-three-seg-mbuf.svg
 create mode 100644 lib/librte_gso/Makefile
 create mode 100644 lib/librte_gso/gso_common.c
 create mode 100644 lib/librte_gso/gso_common.h
 create mode 100644 lib/librte_gso/gso_tcp4.c
 create mode 100644 lib/librte_gso/gso_tcp4.h
 create mode 100644 lib/librte_gso/gso_tunnel_tcp4.c
 create mode 100644 lib/librte_gso/gso_tunnel_tcp4.h
 create mode 100644 lib/librte_gso/rte_gso.c
 create mode 100644 lib/librte_gso/rte_gso.h
 create mode 100644 lib/librte_gso/rte_gso_version.map


More information about the dev mailing list