[dpdk-dev] [PATCH RFC 1/6] mbuf: update mbuf structure for QinQ support

[Zhang, Helin]

> -----Original Message-----
> From: Richardson, Bruce
> Sent: Wednesday, May 6, 2015 4:39 PM
> To: Zhang, Helin
> Cc: Ananyev, Konstantin; dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH RFC 1/6] mbuf: update mbuf structure for
> QinQ support
> 
> On Wed, May 06, 2015 at 04:06:17AM +0000, Zhang, Helin wrote:
> >
> >
> > > -----Original Message-----
> > > From: Ananyev, Konstantin
> > > Sent: Tuesday, May 5, 2015 7:05 PM
> > > To: Zhang, Helin; dev at dpdk.org
> > > Subject: RE: [dpdk-dev] [PATCH RFC 1/6] mbuf: update mbuf structure
> > > for QinQ support
> > >
> > > Hi Helin,
> > >
> > > > -----Original Message-----
> > > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Helin Zhang
> > > > Sent: Tuesday, May 05, 2015 3:32 AM
> > > > To: dev at dpdk.org
> > > > Subject: [dpdk-dev] [PATCH RFC 1/6] mbuf: update mbuf structure
> > > > for QinQ support
> > > >
> > > > To support QinQ, 'vlan_tci' should be replaced by 'vlan_tci0' and
> > > > 'vlan_tci1'. Also new offload flags of 'PKT_RX_QINQ_PKT' and
> > > > 'PKT_TX_QINQ_PKT' should be added.
> > > >
> > > > Signed-off-by: Helin Zhang <helin.zhang at intel.com>
> > > > ---
> > > >  app/test-pmd/flowgen.c                |  2 +-
> > > >  app/test-pmd/macfwd.c                 |  2 +-
> > > >  app/test-pmd/macswap.c                |  2 +-
> > > >  app/test-pmd/rxonly.c                 |  2 +-
> > > >  app/test-pmd/txonly.c                 |  2 +-
> > > >  app/test/packet_burst_generator.c     |  4 ++--
> > > >  lib/librte_ether/rte_ether.h          |  4 ++--
> > > >  lib/librte_mbuf/rte_mbuf.h            | 22
> > > +++++++++++++++++++---
> > > >  lib/librte_pmd_e1000/em_rxtx.c        |  8 ++++----
> > > >  lib/librte_pmd_e1000/igb_rxtx.c       |  8 ++++----
> > > >  lib/librte_pmd_enic/enic_ethdev.c     |  2 +-
> > > >  lib/librte_pmd_enic/enic_main.c       |  2 +-
> > > >  lib/librte_pmd_fm10k/fm10k_rxtx.c     |  2 +-
> > > >  lib/librte_pmd_i40e/i40e_rxtx.c       |  8 ++++----
> > > >  lib/librte_pmd_ixgbe/ixgbe_rxtx.c     | 11 +++++------
> > > >  lib/librte_pmd_vmxnet3/vmxnet3_rxtx.c |  6 +++---
> > > >  16 files changed, 51 insertions(+), 36 deletions(-)
> > > >
> <snip>
> > > --- a/lib/librte_mbuf/rte_mbuf.h
> > > > +++ b/lib/librte_mbuf/rte_mbuf.h
> > > > @@ -101,11 +101,17 @@ extern "C" {  #define
> PKT_RX_TUNNEL_IPV6_HDR
> > > > (1ULL << 12) /**< RX tunnel
> > > packet with IPv6 header. */
> > > >  #define PKT_RX_FDIR_ID       (1ULL << 13) /**< FD id reported
> if
> > > FDIR match. */
> > > >  #define PKT_RX_FDIR_FLX      (1ULL << 14) /**< Flexible bytes
> > > reported if FDIR match. */
> > > > +#define PKT_RX_QINQ_PKT     (1ULL << 15)  /**< RX packet
> with
> > > double VLAN stripped. */
> > > >  /* add new RX flags here */
> > > >
> > > >  /* add new TX flags here */
> > > >
> > > >  /**
> > > > + * Second VLAN insertion (QinQ) flag.
> > > > + */
> > > > +#define PKT_TX_QINQ_PKT    (1ULL << 49)
> > > > +
> > > > +/**
> > > >   * TCP segmentation offload. To enable this offload feature for a
> > > >   * packet to be transmitted on hardware supporting TSO:
> > > >   *  - set the PKT_TX_TCP_SEG flag in mbuf->ol_flags (this flag
> > > > implies @@ -268,7 +274,6 @@ struct rte_mbuf {
> > > >
> > > >  	uint16_t data_len;        /**< Amount of data in segment
> buffer. */
> > > >  	uint32_t pkt_len;         /**< Total pkt len: sum of all
> segments. */
> > > > -	uint16_t vlan_tci;        /**< VLAN Tag Control Identifier (CPU
> > > order) */
> > > >  	uint16_t reserved;
> > >
> > > Now here is an implicit 2-bytes whole between 'reserved' and 'rss'.
> > > Probably better to make it explicit - make 'reserved' uint32_t.
> > Yes, the layout will be changed according to the demands of Vector PMD.
> > The vlan structure will be kept the same, but the mbuf structure
> > layout will be re-organized a bit.
> 
> Why not just put the extra vlan tag into the reserved space. In the original
> work to restructure the mbuf, that was what the reserved space was put
> there for [it was marked as reserved as it was requested that fields not be
> fully dedicated until used, and we did not have double-vlan support at that
> time].
> However, it seems more sensible to put the vlans there now, unless there
> is a good reason to move them to the new location in the mbuf that you
> propose below.
> 
> /Bruce

Thank you very much for the reminder!
The main reason is that we planned to enlarge the packet_type field, so we have to move the vlan fields down. Hopefully the unified packet type can be merged before this one.

Regards,
Helin

> 
> >
> > >
> > > Another thing - it looks like your change will break  ixgbe vector RX.
> > Yes, in the cover-letter, I noted that the vector PMD will be updated
> > soon together with the code changes.
> >
> > >
> > > >  	union {
> > > >  		uint32_t rss;     /**< RSS hash result if RSS enabled */
> > > > @@ -289,6 +294,15 @@ struct rte_mbuf {
> > > >  		uint32_t usr;	  /**< User defined tags. See
> > > rte_distributor_process() */
> > > >  	} hash;                   /**< hash information */
> > > >
> > > > +	/* VLAN tags */
> > > > +	union {
> > > > +		uint32_t vlan_tags;
> > > > +		struct {
> > > > +			uint16_t vlan_tci0;
> > > > +			uint16_t vlan_tci1;
> > >
> > > Do you really need to change vlan_tci to vlan_tci0?
> > > Can't you keep 'vlan_tci' for first vlan tag, and add something like
> > > 'vlan_tci_ext', or 'vlan_tci_next' for second one?
> > > Would save you a lot of changes, again users who use single vlan
> > > wouldn't need to update their code for 2.1.
> > Yes, good point! The names came from the original mbuf definition done
> > by Bruce long long ago. If more guys suggest keeping th old one, and
> > just add a new one, I will do like that in the next version of patch set.
> > Thank you all!
> >
> > >
> > > > +		};
> > > > +	};
> > > > +
> > > >  	uint32_t seqn; /**< Sequence number. See also
> > > > rte_reorder_insert() */
> > > >
> > > >  	/* second cache line - fields only used in slow path or on TX */
> > > > @@
> > > > -766,7 +780,8 @@ static inline void rte_pktmbuf_reset(struct
> > > > rte_mbuf
> > > *m)
> > > >  	m->next = NULL;
> > > >  	m->pkt_len = 0;
> > > >  	m->tx_offload = 0;
> > > > -	m->vlan_tci = 0;
> > > > +	m->vlan_tci0 = 0;
> > > > +	m->vlan_tci1 = 0;
> > >
> > > Why just not:
> > > m-> vlan_tags = 0;
> > > ?
> > Accepted. Good point!
> >
> > >
> > > >  	m->nb_segs = 1;
> > > >  	m->port = 0xff;
> > > >
> > > > @@ -838,7 +853,8 @@ static inline void rte_pktmbuf_attach(struct
> > > rte_mbuf *mi, struct rte_mbuf *m)
> > > >  	mi->data_off = m->data_off;
> > > >  	mi->data_len = m->data_len;
> > > >  	mi->port = m->port;
> > > > -	mi->vlan_tci = m->vlan_tci;
> > > > +	mi->vlan_tci0 = m->vlan_tci0;
> > > > +	mi->vlan_tci1 = m->vlan_tci1;
> > >
> > > Same thing, why not:
> > > mi-> vlan_tags = m-> vlan_tags;
> > > ?
> > Accepted. Good point!
> >
> > Regards,
> > Helin
> 
> <snip>