[dpdk-dev] [RFC] net/mlx4: add TSO support

[Moti Haimovsky]

TCP Segmentation Offload (TSO) is a feature which enables the TCP/IP
network stack to delegate segmentation of a TCP segment to the NIC,
thus saving compute resources.

This RFC proposes to add support for TSO to the MLX4 PMD.

Prerequisites:
In order for the PMD to recognize the TSO capabilities of the device
one has to use:
* RDMA-core v18.0 or above.
* Linux kernel 4.16 or above.

Assumptions:
* mlx4 PMD will follow the TSO support implemented in mlx5 PMD. 
* PMD is backwards compatible.
  ** The PMD will continue work with the kernels and RDMA-core
     supported by it today.
  ** The PMD will continue to work with devices not supporting TSO. 

Changes proposed in the PMD for implementing TSO:
* At init, query the device for TSO support and MAX segment size
  being supported.
  This will also determine if the PMD will advertise support for TSO
  (dev_info->tx_offload_capa |= DEV_TX_OFFLOAD_TCP_TSO;)
* Calling create-qp when creating a Tx queue will have to consider
  the MAX TSO header size when calculating the actual queue buffer
  size. This may be abstracted by calling ibv_create_qp_ex with
  IBV_QP_INIT_ATTR_MAX_TSO_HEADER as comp flag rather than
  ibv_create_qp.
  If this breaks backwards compatibility then this calculation will
  be done in the PMD code.
* Modify tx_burst function to:
  **  Check for TSO flag indication in the packets of the packet burst
      (buf->ol_flags & PKT_TX_TCP_SEG).
  **  For TSO packet create the WQE appropriate for sending a TSO packet
      and fill it with packet info and  L2/L3/L4 Headers.
* Modify Tx completion function to handle releasing of TSO packet
  buffers that were transmitted.

Concerns:
* Impact of changing Tx send routine on performance.
  The performance of the tx_burst routine for non-TSO packets may be
  affected just by placing the code that handles TSO packets in it,
  so we may want to consider having a dedicated routine for TSO packets.
* No MAX-TSO parameter.
   This is a cross-PMD issue that may need a separate mailing thread to handle.
   As for today there is no way for the PMD to advertise the MAX-TSO
   it or its HW support as done with other capabilities.
   (The indirection table size for example.
    see rte_eth_dev_info.reta_size in rte_ethdev.h).
   Also there is no DPDK parameter or constant value that the PMD
   can use in order to know the MAX-TSO the system requires.
   This prevents applications from determining the MAX-TSO that can be
   used leading to configuration mismatches that may lead to transmit
   failures or to less-than-optimize TSO configuration in the best case.
   I propose to add a max_tso field in rte_eth_dev_info that will allow
   the PMD to advertise the max tso is supports. This can be used by
   DPDK applications to determine what TSO size to use.
   If this is a major change that cannot fit the 18.08 schedule then
   I propose to add a MAX_TSO constant in rte_ethdev.h, The PMD will
   compare this value whit its own MAX-TSO and if it cannot meet the
   defined value it will not advertise that it is a TSO capable device.
* Handling packets longer then MAX-TSO
   In case a PMD is requested to send a TSO packet which is longer than
   MAX-TSO the PMD send routine should return with an error.
   A different approach that can be used on the future is to apply GSO
   to those packets using the GSO lib in DPDK.

I am interested in general design comments and concerns listed above.
    
Signed-off-by: Moti Haimovsky <motih at mellanox.com>

-- 
1.8.3.1