[dpdk-dev] [PATCH 0/5] Support TCP/IPv4, VxLAN and GRE GSO in DPDK

[Jiayu Hu]

Hi Konstantin,

Thanks for your suggestions. Feedbacks are inline.

Thanks,
Jiayu

On Wed, Aug 30, 2017 at 09:37:42AM +0800, Ananyev, Konstantin wrote:
> 
> Hi Jiayu,
> Few questions/comments from me below in in next few mails.
> Thanks
> Konstantin
> 
> > 
> > Generic Segmentation Offload (GSO) is a SW technique to split large
> > packets into small ones. Akin to TSO, GSO enables applications to
> > operate on large packets, thus reducing per-packet processing overhead.
> > 
> > To enable more flexibility to applications, DPDK GSO is implemented
> > as a standalone library. Applications explicitly use the GSO library
> > to segment packets. This patch adds GSO support to DPDK for specific
> > packet types: specifically, TCP/IPv4, VxLAN, and GRE.
> > 
> > The first patch introduces the GSO API framework. The second patch
> > adds GSO support for TCP/IPv4 packets (containing an optional VLAN
> > tag). The third patch adds GSO support for VxLAN packets that contain
> > outer IPv4, and inner TCP/IPv4 headers (plus optional inner and/or
> > outer VLAN tags). The fourth patch adds GSO support for GRE packets
> > that contain outer IPv4, and inner TCP/IPv4 headers (with optional
> > outer VLAN tag). The last patch in the series enables TCP/IPv4, VxLAN,
> > and GRE GSO in testpmd's checksum forwarding engine.
> > 
> > The performance of TCP/IPv4 GSO on a 10Gbps link is demonstrated using
> > iperf. Setup for the test is described as follows:
> > 
> > a. Connect 2 x 10Gbps physical ports (P0, P1), together physically.
> > b. Launch testpmd with P0 and a vhost-user port, and use csum
> >    forwarding engine.
> > c. Select IP and TCP HW checksum calculation for P0; select TCP HW
> >    checksum calculation for vhost-user port.
> > d. Launch a VM with csum and tso offloading enabled.
> > e. Run iperf-client on virtio-net port in the VM to send TCP packets.
> 
> Not sure I understand the setup correctly:
> So testpmd forwards packets between P0 and vhost-user port, right?

Yes.

> And who uses P1? iperf-server over linux kernel?

P1 is possessed by linux kernel.

> Also is P1 on another box or not?

P0 and P1 are in the same machine and are connected physically.

> 
> > 
> > With GSO enabled for P0 in testpmd, observed iperf throughput is ~9Gbps.
> 
> Ok, and if GSO is disabled what is the throughput?
> Another stupid question: if P0 is physical 10G (ixgbe?) we can just enable a TSO on it, right?
> If so, what would be the TSO numbers here?

Here are more detailed experiment information:

test1: only enable GSO for p0, GSO size is 1518, use two iperf-clients (i.e. "-P 2")
test2: only enable TSO for p0, TSO size is 1518, use two iperf-clients
test3: disable TSO and GSO, use two iperf-clients

test1 performance: 8.6Gpbs
test2 throughput: 9.5Gbps
test3 throughput: 3Mbps

> 
> In fact, could you probably explain a bit more, what supposed to be a main usage model for that library?

The GSO library is just a SW segmentation method, which can be used by applications, like OVS.
Currently, most of NICs supports to segment TCP and UDP packets, but not for all NICs. So current
OVS doesn't enable TSO, as a result of lacking a SW segmentation fallback. Besides, the protocol
types in HW segmentation are limited. So it's necessary to provide a SW segmentation solution.

With the GSO library, OVS and other applications are able to receive large packets from VMs and
process these large packets, instead of standard ones (i.e. 1518B). So the per-packet overhead is
reduced, since the number of packets needed processing is much fewer.

> Is that to perform segmentation on (virtual) devices that doesn't support HW TSO or ...?

When launch qemu with enabling TSO or GSO, the virtual device doesn't really do segmentation.
It directly sends large packets. Therefore, testpmd can receive large packets from the VM and
then perform GSO. The GSO/TSO behavior of virtual devices is different from physical NICs.

> Again would it be for a termination point (packets were just formed and filled) by the caller,
> or is that for box in the middle which just forwards packets between nodes?
> If the later one, then we'll probably already have most of our packets segmented properly, no?
>   
> > The experimental data of VxLAN and GRE will be shown later.
> > 
> > Jiayu Hu (3):
> >   lib: add Generic Segmentation Offload API framework
> >   gso/lib: add TCP/IPv4 GSO support
> >   app/testpmd: enable TCP/IPv4, VxLAN and GRE GSO
> > 
> > Mark Kavanagh (2):
> >   lib/gso: add VxLAN GSO support
> >   lib/gso: add GRE GSO support
> > 
> >  app/test-pmd/cmdline.c                  | 121 +++++++++
> >  app/test-pmd/config.c                   |  25 ++
> >  app/test-pmd/csumonly.c                 |  68 ++++-
> >  app/test-pmd/testpmd.c                  |   9 +
> >  app/test-pmd/testpmd.h                  |  10 +
> >  config/common_base                      |   5 +
> >  lib/Makefile                            |   2 +
> >  lib/librte_eal/common/include/rte_log.h |   1 +
> >  lib/librte_gso/Makefile                 |  52 ++++
> >  lib/librte_gso/gso_common.c             | 431 ++++++++++++++++++++++++++++++++
> >  lib/librte_gso/gso_common.h             | 180 +++++++++++++
> >  lib/librte_gso/gso_tcp.c                |  82 ++++++
> >  lib/librte_gso/gso_tcp.h                |  73 ++++++
> >  lib/librte_gso/gso_tunnel.c             |  62 +++++
> >  lib/librte_gso/gso_tunnel.h             |  46 ++++
> >  lib/librte_gso/rte_gso.c                | 100 ++++++++
> >  lib/librte_gso/rte_gso.h                | 122 +++++++++
> >  lib/librte_gso/rte_gso_version.map      |   7 +
> >  mk/rte.app.mk                           |   1 +
> >  19 files changed, 1392 insertions(+), 5 deletions(-)
> >  create mode 100644 lib/librte_gso/Makefile
> >  create mode 100644 lib/librte_gso/gso_common.c
> >  create mode 100644 lib/librte_gso/gso_common.h
> >  create mode 100644 lib/librte_gso/gso_tcp.c
> >  create mode 100644 lib/librte_gso/gso_tcp.h
> >  create mode 100644 lib/librte_gso/gso_tunnel.c
> >  create mode 100644 lib/librte_gso/gso_tunnel.h
> >  create mode 100644 lib/librte_gso/rte_gso.c
> >  create mode 100644 lib/librte_gso/rte_gso.h
> >  create mode 100644 lib/librte_gso/rte_gso_version.map
> > 
> > --
> > 2.7.4