[dpdk-dev] [PATCH v2 1/5] ethdev: introduce Tx generic tunnel offloads

[Xueming(Steven) Li]

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Xueming(Steven) Li
> Sent: Wednesday, January 31, 2018 1:54 AM
> To: Ananyev, Konstantin <konstantin.ananyev at intel.com>; Olivier MATZ
> <olivier.matz at 6wind.com>
> Cc: dev at dpdk.org; Wu, Jingjing <jingjing.wu at intel.com>; Shahaf Shuler
> <shahafs at mellanox.com>; Yongseok Koh <yskoh at mellanox.com>; Thomas Monjalon
> <thomas at monjalon.net>; Yigit, Ferruh <ferruh.yigit at intel.com>
> Subject: Re: [dpdk-dev] [PATCH v2 1/5] ethdev: introduce Tx generic tunnel
> offloads
> 
> 
> 
> > -----Original Message-----
> > From: Ananyev, Konstantin [mailto:konstantin.ananyev at intel.com]
> > Sent: Wednesday, January 31, 2018 1:05 AM
> > To: Xueming(Steven) Li <xuemingl at mellanox.com>; Olivier MATZ
> > <olivier.matz at 6wind.com>
> > Cc: dev at dpdk.org; Wu, Jingjing <jingjing.wu at intel.com>; Shahaf Shuler
> > <shahafs at mellanox.com>; Yongseok Koh <yskoh at mellanox.com>; Thomas
> > Monjalon <thomas at monjalon.net>; Yigit, Ferruh <ferruh.yigit at intel.com>
> > Subject: RE: [dpdk-dev] [PATCH v2 1/5] ethdev: introduce Tx generic
> > tunnel offloads
> >
> >
> >
> > > -----Original Message-----
> > > From: Xueming(Steven) Li [mailto:xuemingl at mellanox.com]
> > > Sent: Tuesday, January 30, 2018 4:10 PM
> > > To: Ananyev, Konstantin <konstantin.ananyev at intel.com>; Olivier MATZ
> > > <olivier.matz at 6wind.com>
> > > Cc: dev at dpdk.org; Wu, Jingjing <jingjing.wu at intel.com>; Shahaf
> > > Shuler <shahafs at mellanox.com>; Yongseok Koh <yskoh at mellanox.com>;
> > > Thomas Monjalon <thomas at monjalon.net>; Yigit, Ferruh
> > > <ferruh.yigit at intel.com>
> > > Subject: RE: [dpdk-dev] [PATCH v2 1/5] ethdev: introduce Tx generic
> > > tunnel offloads
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Ananyev, Konstantin [mailto:konstantin.ananyev at intel.com]
> > > > Sent: Wednesday, January 31, 2018 12:03 AM
> > > > To: Xueming(Steven) Li <xuemingl at mellanox.com>; Olivier MATZ
> > > > <olivier.matz at 6wind.com>
> > > > Cc: dev at dpdk.org; Wu, Jingjing <jingjing.wu at intel.com>; Shahaf
> > > > Shuler <shahafs at mellanox.com>; Yongseok Koh <yskoh at mellanox.com>;
> > > > Thomas Monjalon <thomas at monjalon.net>; Yigit, Ferruh
> > > > <ferruh.yigit at intel.com>
> > > > Subject: RE: [dpdk-dev] [PATCH v2 1/5] ethdev: introduce Tx
> > > > generic tunnel offloads
> > > >
> > > >
> > > >
> > > > > > > >
> > > > > > > > > > > This patch introduce new TX offloads flag for
> > > > > > > > > > > devices that support tunnel agnostic checksum and TSO
> offloads.
> > > > > > > > > > >
> > > > > > > > > > > The support from the device is for inner and outer
> > > > > > > > > > > checksums on IPV4/TCP/UDP and TSO for *any packet
> > > > > > > > > > > with the following
> > > > > > format*:
> > > > > > > > > > >
> > > > > > > > > > > < some headers > / [optional IPv4/IPv6] / [optional
> > > > > > > > > > > TCP/UDP] / <some
> > > > > > > > > > > headers> / [optional inner IPv4/IPv6] / [optional
> > > > > > > > > > > headers> TCP/UDP]
> > > > > > > > > > >
> > > > > > > > > > > For example the following packets can use this feature:
> > > > > > > > > > >
> > > > > > > > > > > 1. eth / ipv4 / udp / VXLAN / ip / tcp 2. eth / ipv4
> > > > > > > > > > > / GRE / MPLS /
> > > > > > > > > > > ipv4 / udp
> > > > > > > > > >
> > > > > > > > > > So in terms of usage - what is the difference with
> > > > > > > > > > current TSO
> > > > > > types?
> > > > > > > > > > Konstantin
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Traditionally, HW only recognize "known" tunnel type, do
> > > > > > > > > TSO calculation based on L3/L4 headers known to tunnel
> > > > > > > > > type. For example, it must be
> > > > > > > > > L2 header after VXLAN, then L3. While this Generic
> > > > > > > > > offloading provides inner/outer L3/L4 header info(len
> > > > > > > > > and
> > > > > > > > > offset) to HW, and thus tunnel info become less important.
> > > > > > > > > Please note the MPLS over GRE tunnel in last example above.
> > > > > > > >
> > > > > > > > Ok, but I wonder when the user would like to do TSO on
> > > > > > > > tunnel packet, for this offload - would he need to do
> > > > > > > > something differently from what he has to do now:
> > > > > > > > raise PKT_TX_TCP_SEG and related flags, raise appropriate
> > > > > > > > PKT_TX_TUNNEL_* flag, fill l2_len, l3_len,
> > > > > > l4_len,tso_segsz,outer_l2_len,outer_l3_len?
> > > > > > > >
> > > > > > >
> > > > > > > Yes, these fields are sufficient except PKT_TX_TUNNEL_*,
> > > > > > > major target of this new feature is to support "unknown"
> > > > > > > tunnel offloading, it
> > > > > > supports "known"
> > > > > > > tunnel type as well.
> > > > > >
> > > > > > Ok, but user would still need to set some flag to indicate
> > > > > > that this is a tunnel packet, and he wants TSO over it, right?
> > > > > > For pre-defined tunnel types it can be one of PKT_TX_TUNNEL_*
> > > > > > (which actually means that user still have to know tunnel type
> > > > > > anyway?) But for some not defined tunnel type - what it would be?
> > > > > > Konstantin
> > > > > >
> > > > >
> > > > > As this feature target to TX path, Outer length as tunnel
> > > > > indication, leave it empty if tunnel not defined.
> > > >
> > > > Sorry, I didn't get it.
> > > > We need to let PMD know that it is a tunnel packet, right?
> > > > So we do need to raise PKT_TX_TUNNEL_* flag.
> > > >
> > >
> > > In my current code, mbuf.outer_l2_len is used to test tunnel packet.
> > > Agree a new tunnel flag would be better.
> > >
> > > > >
> > > > > But I think it good to define something like:
> > > > >  	PKT_TX_TUNNEL_GENERIC = PKT_TX_TUNNEL_MASK
> > > >
> > > > Yes, that's an option, I would probably name it
> PKT_TX_TUNNEL_UNKNOWN.
> > > >
> > > > > And a new flag PKT_TX_OUTER_UDP, how do you think?
> > > >
> > > > Not sure why do we need it?
> > > > HW still needs to know outer_l4_type to be able to work correctly?
> > >
> > > For tunnel type like vxlan, if outer UDP present, hw has to update
> > > UDP length field for each TSO packet segment.
> >
> > I understand that, but I thought that HW is smart enough to parse the
> > header and recognize outer L3/L4 type  - as it is a 'generic'
> > tunneling (sorry I am not familiar with MLX HW at all).
> 
> It might be useful if the outer encapsulation not regular, for example
> MPLS.
> 
> > From what you saying - that assumption was wrong and user still  need
> > to provide some packet-type info at  least about outer headers, right?
> > So what else need to be set?
> > Probably PKT_TX_OUTER_IPV*, might be something else?
> 
> Sorry for the confusion, besides optional outer UDP type, still need
> PKT_TX_IPV4/6 and PKT_TX_OUTER_IPV4/6

About introducing PKT_TX_OUTER_UDP, mbuf has defined packet_type which
has all inner/outer and tunnel types, not sure whether it good to
request user set packet_type field, normally it used on RX path.

> 
> 
> > Konstantin
> >
> > > Konstantin
> > > >
> > > > >
> > > > > > >
> > > > > > > PKT_TX_TUNNEL_VXLAN has to be used as a hint if outer UDP
> > expected.
> > > > > > >
> > > > > > > > Konstantin
> > > > > > > >
> > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Signed-off-by: Xueming Li <xuemingl at mellanox.com>
> > > > > > > > > > > ---
> > > > > > > > > > >  lib/librte_ether/rte_ethdev.h | 10 ++++++++++
> > > > > > > > > > >  1 file changed, 10 insertions(+)
> > > > > > > > > > >
> > > > > > > > > > > diff --git a/lib/librte_ether/rte_ethdev.h
> > > > > > > > > > > b/lib/librte_ether/rte_ethdev.h index
> > > > > > > > > > > 1a5b4cdc5..d8d08ccb2
> > > > > > > > > > > 100644
> > > > > > > > > > > --- a/lib/librte_ether/rte_ethdev.h
> > > > > > > > > > > +++ b/lib/librte_ether/rte_ethdev.h
> > > > > > > > > > > @@ -979,6 +979,16 @@ struct rte_eth_conf {
> > > > > > > > > > >   *   the same mempool and has refcnt = 1.
> > > > > > > > > > >   */
> > > > > > > > > > >  #define DEV_TX_OFFLOAD_SECURITY         0x00020000
> > > > > > > > > > > +/**< Device supports generic tunnel checksum and
> > > > > > > > > > > +TSO
> > > > offloading.
> > > > > > > > > > > + * Checksum and TSO are done based on following
> > > > > > > > > > > + mbuf
> > fields:
> > > > > > > > > > > + *   - Length of each header
> > > > > > > > > > > + *   - Type of outer/inner L3 type, IPv4 or IPv6
> > > > > > > > > > > + *   - Type of outer/inner L4 type, TCP or UDP.
> > > > > > > > > > > + *	- PKT_TX_TUNNEL_VXLAN implies outer UDP type.
> > > > > > > > > > > + *	- PKT_TX_TCP_SEG implies inner TCP type.
> > > > > > > > > > > + * Tunnel type is optional except
> > > > > > > > > > > +PKT_TX_TUNNEL_VXLAN to hint outer
> > > > > > > > UDP.
> > > > > > > > > > > + */
> > > > > > > > > > > +#define DEV_TX_OFFLOAD_GENERIC_TNL_CKSUM_TSO
> > 	0x00040000
> > > > > > > > > > >
> > > > > > > > > > >  /*
> > > > > > > > > > >   * If new Tx offload capabilities are defined, they
> > > > > > > > > > > also must be
> > > > > > > > > > > --
> > > > > > > > > > > 2.13.3