[dpdk-dev] cryptodev - Session and queue pair relationship

[Declan Doherty]

On 06/02/17 13:35, Akhil Goyal wrote:
> Hi,
>
Hey Akhil, see my thoughts inline

> I have some issues w.r.t the mapping sessions and queue pairs.
>
> As per my understanding:
> - Number of sessions may be large – they are independent of number of
> queue pairs

Yes, cryptodev assumes no implicit connection between sessions and
queue pairs, the current PMDs just use the crypto session to store the 
immutable data (keys etc) for a particular crypto transform or chain of 
transforms in a format specific to that PMD with no statefull information.

> - Queue pairs are L-core specific

Not exactly, queue pairs like ethdev queues are not thread safe, so we
assume that only a single l-core will be using a queue pair at any time
unless the application layer has introduce a locking mechanism to 
provide thread safety.

> - Depending on the implementation, one queue pair can be mapped to many
> sessions. Or, Only one queue pair for every session- especially in the
> systems having large number of queues (hw).

Currently none of the software crypto PMDs or Intel QuickAssist hardware
accelerated PMD make any assumptions regarding coupling/mapping of 
sessions to queue pairs, so today a users could freely change the queue
pair which a session is processed on, or even go as far using the  ame 
session for processing on different queue simultaneously as the sessions 
are stateless, obviously this could introduce issues for statefull 
higher level protocol using the cryptodev PMD service but the cryptodev 
API doesn't prohibit this usage model.


> - Sessions can be created on the fly – typical rekeying use-cases.
> Generally done by the control threads.
>

Sure, there is no restriction on session creation other than an element 
being free in the mempool which the session is being created on.

> There seems to be no straight way for the underlying driver
> implementation to know, what all sessions are mapped to a particular
> queue pair. The session and queue pair information is first time exposed
> in the enqueue command.
>
> One of the NXP Crypto Hardware drivers uses per session data structures
> (descriptors) which need to be configured for hardware queues.  Though
> this information can be extracted from the first enqueue command for a
> particular session, it will add checks in the data path. Also, it will
> bring down the connection setup rate.

We haven't had to support this model of coupling sessions to queue pairs
in any PMDs before. If I understand correctly, in the hardware model you
need to support a queue pair can only be configured to support the 
processing of a single session at any one time and it only supports that 
session until it is reconfigured, is this correct? So if a session needs 
to be re-keyed the queue pair would need to be reconfigured?

>
> In the API rte_cryptodev_sym_session_create(), we create session on a
> particular device, but there is no information of queue pair being shared.
>
> 1. We want to propose to change the session create/config API to also
> take queue pair id as argument.
> struct rte_cryptodev_sym_session *
> rte_cryptodev_sym_session_create(uint8_t dev_id,
>                               struct rte_crypto_sym_xform *xform) to
> also take “uint16_t qp;”
>
> This will also return “in-use” error, if the underlying hardware only
> support 1 session/descriptor per qp.

I my mind the idea of coupling the session_create function to the queue
pair of a device doesn't feel right as it would certainly put 
unnecessary constraint on all existing PMDs queue pairs.

One possible approach would be to extend the the queue_pair_setup 
function to take an opaque parameter which would allow you to pass a 
session through and would be  an approach more in keeping with the 
cryptodev current model, but you would then still need to verify that 
the operations being enqueued have the same session as the configured 
device, assuming that the packet are being enqueued from the host.

If you need to re-key or change the session you could re-initialize the 
queue pair while the device is still active, but stopping the queue pair.

Following a sequence something like:
stop_qp()
setup_qp()
start_qp()


Another option Fiona suggested would be to add 2 new APIs

rte_cryptodev_queue_pair_attach_sym_session/queue_pair_detach_sym_session 
this would allow dynamic attaching of one or more sessions to device if 
it supported this sort of static mapping of sessions to queue pairs.


>
> 2. Currently the application configures the *nb_descriptors* in the
> *rte_cryptodev_queue_pair_setup*. Should we add the queue pair
> capability API?
>

Regarding capabilities, I think this should be just propagated through 
the device capabilities, something like a max number of session mapped 
per queue pair, which would be zero for all/most current devices, and 
could be 1 or greater for your device. This is assuming that all queue 
pairs can all support the same crypto transforms capabilities and that 
different queue pairs have different capabilities which could get very 
messy to discover.

>
> Please share your feedback, I will submit the patch accordingly.
>
> Regards,
> Akhil
>
>
>