[dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Ananyev, Konstantin]

Hi Fiona,

> Hi Shally, Ahmed, and anyone else interested in compressdev,
> 
> I mentioned last week that we've been exploring using something other than mbufs to pass src/dst buffers to compressdev PMDs.
> 
> Reasons:
>  - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
>     can be greater and it would add cycles to have to break up into smaller segments.
>  - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
>     originate in other data structures.
>  - There's a 2 cache-line overhead for every segment in a chain, most of this data
>     is network-related, not needed by compressdev
> So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size and give more flexibility if
> compressdev ever needs any comp-specific meta-data.
> 
> We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is commonly used by storage
> applications. This would replace the src and dest mbufs in the  op.
> I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review comments there.
> https://github.com/pablodelara/dpdk-draft-compressdev
> Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may see it on github.
> 
> Note : We also considered having no data structures in the op, instead the application
> would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
> for each next segment as needed. While this is quite flexible and allow the application
> to keep its data in its native structures, it's likely to cost more cycles.

As I said in different thread it will not only slowdown things, but will make it difficult
(if possible at all) for compressdev PMDs to support DPDK MP model.
Konstantin

> So we're not proposing this at the moment, but hope to benchmark it later while the API is still experimental.
> 
> General feedback on direction is welcome here on the mailing list.
> For feedback on the details of implementation we would appreciate comments on github.
> 
> Regards,
> Fiona.