[dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

[Verma, Shally]

@Trahe, Fiona>> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
>> that they're not optimal for storage and we may propose changes in 18.08.
[Shally] Sounds good to us too.

@Ahmed Mansour . I am assuming that transferring from mbuf to regular buffers and back does
>not involve some time consuming work like data copying and such.
[Shally] I too assume copying shouldn't be a need and a big no-no. We normally extract and pass buf_addr from mbuf as it is to HW.
So implicit assumption is data memory is dma-able to device.

Thanks
Shally

>-----Original Message-----
>From: Ahmed Mansour [mailto:ahmed.mansour at nxp.com]
>Sent: 15 March 2018 00:32
>To: Trahe, Fiona <fiona.trahe at intel.com>; Verma, Shally <Shally.Verma at cavium.com>; dev at dpdk.org
>Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
>Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila <Sunila.Sahu at cavium.com>; Challa, Mahipal
><Mahipal.Challa at cavium.com>; Jain, Deepak K <deepak.k.jain at intel.com>; Hemant Agrawal <hemant.agrawal at nxp.com>; Roy
>Pledge <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; Daly, Lee <lee.daly at intel.com>; Jozwiak, TomaszX
><tomaszx.jozwiak at intel.com>; Alok Makhariya <alok.makhariya at nxp.com>; Shreyansh Jain <shreyansh.jain at nxp.com>
>Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>
>Hi All,
>
>Sticking with mbufs until at least 1805 works for us. We also see
>storage as the main use case, but ipcomp maybe an important customer use
>case in the future. Nonetheless, I see the mbuf formatting as inherently
>external to the compressdev APIs. An application doing ipcomp should
>just do the mbuf packaging outside of compressdev. At least that is what
>current software implementation of ipcomp do when using zlib.net. I am
>assuming that transferring from mbuf to regular buffers and back does
>not involve some time consuming work like data copying and such.
>
>Thanks,
>
>Ahmed
>
>On 3/14/2018 2:39 PM, Trahe, Fiona wrote:
>> Hi Shally, Ahmed, et al,
>>
>> Following internal and community feedback we've decided that there's still too much churn in this.
>> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
>> that they're not optimal for storage and we may propose changes in 18.08.
>> Compressdev will start as an experimental API in 18.05 - we'll POC and benchmark alternatives
>> or API extensions once we get time to do so.
>>
>> Regards,
>> Fiona
>>
>>> -----Original Message-----
>>> From: Verma, Shally [mailto:Shally.Verma at cavium.com]
>>> Sent: Wednesday, March 14, 2018 12:51 PM
>>> To: Trahe, Fiona <fiona.trahe at intel.com>; Ahmed Mansour <ahmed.mansour at nxp.com>; dev at dpdk.org
>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
>>> <NarayanaPrasad.Athreya at cavium.com>; Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila
>>> <Sunila.Sahu at cavium.com>; Challa, Mahipal <Mahipal.Challa at cavium.com>; Jain, Deepak K
>>> <deepak.k.jain at intel.com>; Hemant Agrawal <hemant.agrawal at nxp.com>; Roy Pledge
>>> <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; Daly, Lee <lee.daly at intel.com>;
>>> Jozwiak, TomaszX <tomaszx.jozwiak at intel.com>
>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>
>>>
>>>
>>>> -----Original Message-----
>>>> From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
>>>> Sent: 13 March 2018 21:22
>>>> To: Verma, Shally <Shally.Verma at cavium.com>; Ahmed Mansour <ahmed.mansour at nxp.com>;
>>> dev at dpdk.org
>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
>>> <NarayanaPrasad.Athreya at cavium.com>;
>>>> Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila <Sunila.Sahu at cavium.com>; Challa, Mahipal
>>>> <Mahipal.Challa at cavium.com>; Jain, Deepak K <deepak.k.jain at intel.com>; Hemant Agrawal
>>> <hemant.agrawal at nxp.com>; Roy
>>>> Pledge <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; Daly, Lee
>>> <lee.daly at intel.com>; Jozwiak, TomaszX
>>>> <tomaszx.jozwiak at intel.com>; Trahe, Fiona <fiona.trahe at intel.com>
>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>
>>>> Hi Shally,
>>>>
>>>>> -----Original Message-----
>>>>> From: Verma, Shally [mailto:Shally.Verma at cavium.com]
>>>>> Sent: Tuesday, March 13, 2018 8:15 AM
>>>>> To: Trahe, Fiona <fiona.trahe at intel.com>; Ahmed Mansour <ahmed.mansour at nxp.com>;
>>> dev at dpdk.org
>>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
>>>>> <NarayanaPrasad.Athreya at cavium.com>; Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila
>>>>> <Sunila.Sahu at cavium.com>; Challa, Mahipal <Mahipal.Challa at cavium.com>; Jain, Deepak K
>>>>> <deepak.k.jain at intel.com>; Hemant Agrawal <hemant.agrawal at nxp.com>; Roy Pledge
>>>>> <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; fiona.trahe at gmail.com; Daly, Lee
>>>>> <lee.daly at intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak at intel.com>
>>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>>
>>>>> HI Fiona
>>>>>
>>>>> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline
>>> overhead
>>>>> and their more suitability to n/w applications. Given that, I understand benefit of having another
>>> structure
>>>>> to input data but then what is proposal for ipcomp like application where mbuf usage may be a better
>>>>> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate
>>> data
>>>>> structure depending on their requirement.
>>>> [Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in the
>>>> compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(),
>>>> rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
>>>> For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
>>>> We see storage applications rather than IPComp as the main use-case for compressdev, so would prefer
>>>> to optimise for that.
>>>> Do you think otherwise?
>>> [Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps. So, we envision mbuf support
>>> as necessary. So, I think we can add two APIs one which process on rte_comp_op and other on rte_mbufs
>>> to make it simpler.
>>>
>>>>> Further comments, on github.
>>>>>
>>>>> Thanks
>>>>> Shally
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
>>>>>> Sent: 12 March 2018 21:31
>>>>>> To: Ahmed Mansour <ahmed.mansour at nxp.com>; Verma, Shally <Shally.Verma at cavium.com>;
>>>>> dev at dpdk.org
>>>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
>>>>> <NarayanaPrasad.Athreya at cavium.com>;
>>>>>> Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila <Sunila.Sahu at cavium.com>; Challa,
>>> Mahipal
>>>>>> <Mahipal.Challa at cavium.com>; Jain, Deepak K <deepak.k.jain at intel.com>; Hemant Agrawal
>>>>> <hemant.agrawal at nxp.com>; Roy
>>>>>> Pledge <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; fiona.trahe at gmail.com;
>>> Daly,
>>>>> Lee <lee.daly at intel.com>;
>>>>>> Jozwiak, TomaszX <tomaszx.jozwiak at intel.com>
>>>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
>>>>>>
>>>>>> Hi Shally, Ahmed, and anyone else interested in compressdev,
>>>>>>
>>>>>> I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
>>>>> buffers to compressdev PMDs.
>>>>>> Reasons:
>>>>>> - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
>>>>>>    can be greater and it would add cycles to have to break up into smaller segments.
>>>>>> - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
>>>>>>    originate in other data structures.
>>>>>> - There's a 2 cache-line overhead for every segment in a chain, most of this data
>>>>>>    is network-related, not needed by compressdev
>>>>>> So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1 size
>>> and
>>>>> give more flexibility if
>>>>>> compressdev ever needs any comp-specific meta-data.
>>>>>>
>>>>>> We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which is
>>>>> commonly used by storage
>>>>>> applications. This would replace the src and dest mbufs in the  op.
>>>>>> I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
>>>>> comments there.
>>>>>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpablodelara%2Fdpdk-draft-
>compressdev&data=02%7C01%7Cahmed.mansour%40nxp.com%7C6a8977f9b3714d58621708d589dae567%7C686ea1d3bc2b4c6fa92cd
>99c5c301635%7C0%7C0%7C636566495639618830&sdata=wmFrxeUNyXdxI5%2Fp5gCmyIRfeDnbHebBJXbztqdsMrc%3D&reserved=0
>>>>>> Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than may
>>> see
>>>>> it on github.
>>>>>> Note : We also considered having no data structures in the op, instead the application
>>>>>> would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
>>>>>> for each next segment as needed. While this is quite flexible and allow the application
>>>>>> to keep its data in its native structures, it's likely to cost more cycles.
>>>>>> So we're not proposing this at the moment, but hope to benchmark it later while the API is still
>>>>> experimental.
>>>>>> General feedback on direction is welcome here on the mailing list.
>>>>>> For feedback on the details of implementation we would appreciate comments on github.
>>>>>>
>>>>>> Regards,
>>>>>> Fiona.
>