[dpdk-dev] [RFC][PATCH 0/5] cryptodev: Adding support for inline crypto processing of IPsec flows

[Boris Pismenny]

> >> 5. The addition of inline crypto metadata into the rte_mbuf structure to
> allow the required egress metadata to be given to the NIC PMD to build the
> necessary transmit descriptors in tx_burst processing when the
> PKT_TX_IPSEC_INLINE_CRYPTO is set. We are looking for feedback on a
> better approach to handling the passing of this metadata to the NIC as it is
> understood that different hardware accelerators which support this offload
> may have different requirements for metadata depending on
> implementation and other capabilities in the device. One possibility we have
> consider is that the last 16 bytes of mbuf is reserved for device specific
> metadata, which layout is flexible depending on the hardware being used.
> >>
> >> struct rte_mbuf {
> >> ...
> >> 	/** Inline IPSec metadata*/
> >> 	struct {
> >> 	        uint16_t sa_idx;        /**< SA index */
> >> 	        uint8_t  pad_len;       /**< Padding length */
> >> 	        uint8_t  enc;
> >> 	} inline_ipsec;
> >> } __rte_cache_aligned;
> >
> > Assuming that you see the packet with PKT_TX_IPSEC_INLINE_CRYPTO,
> could you infer these parameters from the packet itself?
> >
> 
> In our case this isn't really possible as each packet in a burst could be be
> associated with a different security association/crypto session and will also
> have different lengths/padding etc. We could use some sort of cookie to
> store this, but I think it would have a big performance impact. I do think that
> this structure in the mbuf should not be device specific as it is now for the
> required metadata, but I would like to guarantee that the metadata is in the
> mbuf.
> 
> 
> >
> > This is a nice approach.
> >
> > We are also working on adding support for IPsec inline crypto in DPDK.
> > I hope we could submit a RFC with working code soon.
> Iis your device capable of full IPsec protocol processing, ESP header insertion,
> encap/decap etc? In our case the inline functionality is limited to the crypto
> processing, so we are working on the assumption that the user will be
> integrating with an existing IPsec stack. On ingress the lookup is based on the
> Destination IP and SPI, on egress the metadata is

Currently our device is not capable of full IPsec protocol processing.
But, future devices will not have this limitation and it shouldn't be 
assumed in the API. We also, need to integrate with an existing
IPsec stack. However, we perform a lookup on both egress and
Ingress for source IP, destination IP and SPI.

> 
> >
> > We considered 3 approaches for IPsec inline support:
> > 1. IPsec inline as a cryptodev (like this RFC) 2. IPsec inline as a
> > rte_flow action. (see details below) 3. Mix between approach 2 and
> > approach 3.
> >
> > In approach 2, there is no need for an additional crypto PMD.
> > Inline IPsec is exposed as another feature of a NIC PMD.
> >
> 
> > For the control-path, we introduce a new rte_flow_action_type for
> > crypto and a flag to mark flows as egress flows.
> > Then, it is possible to program the SA by calling rte_flow_create with
> > an appropriate pattern of IP and ESP header fields, and an action that
> > contains rte_crypto_ipsec_xform as the configuration.
> >
> > The main benefit of using the rte_flow API is that we can reuse, the
> > existing API with patterns and actions. For example, it would be
> > possible to add support for UDP encapsulation of IPsec without
> > changing the API. Support for VLAN/VXLAN/GRE/etc could be added
> > similarly to UDP encapsulation.
> 
> This make sense when hw is capable of full offload. So the rte_flow actions
> might be VxLAN and ESP Tunnel for a flow. The other approach is that to
> separate rules are created one for IPsec, then a second for the VxLAN tunnel
> which trigger on the IPsec flow, but this probably implies that either the PMD
> flattens these flow actions into a single action or the hw supports re
> circulation. One concern I would have is population of the namespace of
> rte_flow with IPsec/crypto session material, but I guess it should be possible
> to come up with a clean way of supporting this.

Full offload is not necessary if the device has a capable parser.
Encapsulations could be added by the DPDK application, and
for inline offload the device must be aware of them. Unlike other
crypto PMDs that actually perform encryption, inline crypto PMDs
need to setup the entire packet format up to ESP in order to process
the request. One concern I have in this matter, is that the semantics of
other crypto PMDs are different from inline - they actually perform crypto.
Inline would break if the packet format is not correct.

Adding a VXLAN flow and an ESP tunnel separately is similar to the
third approach, because the PMD requires some indication that it should
flatten these rules and this indication will be in the form of a crypto session.
Maybe this approach will be the best. 

In the 3rd approach there is a crypto PMD that allows for zero changes
in the datapath and it is possible to set device specific metadata in its
enqueue_burst. While, the control path is split between rte_crypto and
rte_flow. First a crypto session is setup, then the crypto session is
provided to rte_flow_create as an action. The crypto session will not
include any networking related code, just the keys and the salt.

What is your concern about the population of rte_flow with IPsec/crypto
material? I think that at least some crypto in rte_flow is necessary to
support advanced use-cases without re-implementing rte_flow inside
rte_crypto.

> 
> 
> > For the data-path, all is handled in the NIC PMD, during rx/tx_burst.
> > While, the application marks the packets for encryption in the
> > transmit path. And it receives packets marked as decrypted/auth-fail
> > on the receive side.
> 
> when you say the application marks the packet, this is essentially the IPsec
> stack? The main benefit in the approach of this RFC is that it is possible to
> integrate the inline crypto processing transparently in the data path. The
> crypto PMD can handle setting and interpreting the metadata required and
> the IPsec stack is just using the crypto PMD as it would any other crypto
> PMD.
> 

Right. I'm concerned about the overhead of going through a crypto PMD,
and I understand that it might be necessary for your device.
Changes in the data path would be minor in any way we implement inline.
If the metadata is not device specific, then shouldn't the application set it
directly based on the SA itself?