[dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative

Trahe, Fiona fiona.trahe at intel.com
Thu Mar 15 10:48:54 CET 2018



> -----Original Message-----
> From: Verma, Shally [mailto:Shally.Verma at cavium.com]
> Sent: Thursday, March 15, 2018 4:12 AM
> To: Ahmed Mansour <ahmed.mansour at nxp.com>; Trahe, Fiona <fiona.trahe at intel.com>; dev at dpdk.org
> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya at cavium.com>; Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila
> <Sunila.Sahu at cavium.com>; Challa, Mahipal <Mahipal.Challa at cavium.com>; Jain, Deepak K
> <deepak.k.jain at intel.com>; Hemant Agrawal <hemant.agrawal at nxp.com>; Roy Pledge
> <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; Daly, Lee <lee.daly at intel.com>;
> Jozwiak, TomaszX <tomaszx.jozwiak at intel.com>; Alok Makhariya <alok.makhariya at nxp.com>; Shreyansh
> Jain <shreyansh.jain at nxp.com>
> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> 
> @Trahe, Fiona>> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs -
> acknowledging
> >> that they're not optimal for storage and we may propose changes in 18.08.
> [Shally] Sounds good to us too.
> 
> @Ahmed Mansour . I am assuming that transferring from mbuf to regular buffers and back does
> >not involve some time consuming work like data copying and such.
> [Shally] I too assume copying shouldn't be a need and a big no-no. We normally extract and pass buf_addr
> from mbuf as it is to HW.
> So implicit assumption is data memory is dma-able to device.
[Fiona] agreed

> 
> Thanks
> Shally
> 
> >-----Original Message-----
> >From: Ahmed Mansour [mailto:ahmed.mansour at nxp.com]
> >Sent: 15 March 2018 00:32
> >To: Trahe, Fiona <fiona.trahe at intel.com>; Verma, Shally <Shally.Verma at cavium.com>; dev at dpdk.org
> >Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya at cavium.com>;
> >Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila <Sunila.Sahu at cavium.com>; Challa, Mahipal
> ><Mahipal.Challa at cavium.com>; Jain, Deepak K <deepak.k.jain at intel.com>; Hemant Agrawal
> <hemant.agrawal at nxp.com>; Roy
> >Pledge <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; Daly, Lee
> <lee.daly at intel.com>; Jozwiak, TomaszX
> ><tomaszx.jozwiak at intel.com>; Alok Makhariya <alok.makhariya at nxp.com>; Shreyansh Jain
> <shreyansh.jain at nxp.com>
> >Subject: Re: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >
> >Hi All,
> >
> >Sticking with mbufs until at least 1805 works for us. We also see
> >storage as the main use case, but ipcomp maybe an important customer use
> >case in the future. Nonetheless, I see the mbuf formatting as inherently
> >external to the compressdev APIs. An application doing ipcomp should
> >just do the mbuf packaging outside of compressdev. At least that is what
> >current software implementation of ipcomp do when using zlib.net. I am
> >assuming that transferring from mbuf to regular buffers and back does
> >not involve some time consuming work like data copying and such.
> >
> >Thanks,
> >
> >Ahmed
> >
> >On 3/14/2018 2:39 PM, Trahe, Fiona wrote:
> >> Hi Shally, Ahmed, et al,
> >>
> >> Following internal and community feedback we've decided that there's still too much churn in this.
> >> We're proposing, in the interest of getting the API out in 18.05, to stick with mbufs - acknowledging
> >> that they're not optimal for storage and we may propose changes in 18.08.
> >> Compressdev will start as an experimental API in 18.05 - we'll POC and benchmark alternatives
> >> or API extensions once we get time to do so.
> >>
> >> Regards,
> >> Fiona
> >>
> >>> -----Original Message-----
> >>> From: Verma, Shally [mailto:Shally.Verma at cavium.com]
> >>> Sent: Wednesday, March 14, 2018 12:51 PM
> >>> To: Trahe, Fiona <fiona.trahe at intel.com>; Ahmed Mansour <ahmed.mansour at nxp.com>;
> dev at dpdk.org
> >>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
> >>> <NarayanaPrasad.Athreya at cavium.com>; Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila
> >>> <Sunila.Sahu at cavium.com>; Challa, Mahipal <Mahipal.Challa at cavium.com>; Jain, Deepak K
> >>> <deepak.k.jain at intel.com>; Hemant Agrawal <hemant.agrawal at nxp.com>; Roy Pledge
> >>> <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; Daly, Lee <lee.daly at intel.com>;
> >>> Jozwiak, TomaszX <tomaszx.jozwiak at intel.com>
> >>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>
> >>>
> >>>
> >>>> -----Original Message-----
> >>>> From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> >>>> Sent: 13 March 2018 21:22
> >>>> To: Verma, Shally <Shally.Verma at cavium.com>; Ahmed Mansour <ahmed.mansour at nxp.com>;
> >>> dev at dpdk.org
> >>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
> >>> <NarayanaPrasad.Athreya at cavium.com>;
> >>>> Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila <Sunila.Sahu at cavium.com>; Challa,
> Mahipal
> >>>> <Mahipal.Challa at cavium.com>; Jain, Deepak K <deepak.k.jain at intel.com>; Hemant Agrawal
> >>> <hemant.agrawal at nxp.com>; Roy
> >>>> Pledge <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; Daly, Lee
> >>> <lee.daly at intel.com>; Jozwiak, TomaszX
> >>>> <tomaszx.jozwiak at intel.com>; Trahe, Fiona <fiona.trahe at intel.com>
> >>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>>
> >>>> Hi Shally,
> >>>>
> >>>>> -----Original Message-----
> >>>>> From: Verma, Shally [mailto:Shally.Verma at cavium.com]
> >>>>> Sent: Tuesday, March 13, 2018 8:15 AM
> >>>>> To: Trahe, Fiona <fiona.trahe at intel.com>; Ahmed Mansour <ahmed.mansour at nxp.com>;
> >>> dev at dpdk.org
> >>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
> >>>>> <NarayanaPrasad.Athreya at cavium.com>; Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu,
> Sunila
> >>>>> <Sunila.Sahu at cavium.com>; Challa, Mahipal <Mahipal.Challa at cavium.com>; Jain, Deepak K
> >>>>> <deepak.k.jain at intel.com>; Hemant Agrawal <hemant.agrawal at nxp.com>; Roy Pledge
> >>>>> <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; fiona.trahe at gmail.com; Daly,
> Lee
> >>>>> <lee.daly at intel.com>; Jozwiak, TomaszX <tomaszx.jozwiak at intel.com>
> >>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>>>
> >>>>> HI Fiona
> >>>>>
> >>>>> So I understand we're moving away from mbufs because of its size limitation (64k) and cacheline
> >>> overhead
> >>>>> and their more suitability to n/w applications. Given that, I understand benefit of having another
> >>> structure
> >>>>> to input data but then what is proposal for ipcomp like application where mbuf usage may be a
> better
> >>>>> option? Should we keep support for both (mbuf and this structure) so that apps can use appropriate
> >>> data
> >>>>> structure depending on their requirement.
> >>>> [Fiona] An application can use pass buffers from an mbuf or mbuf chain to compressdev by filling in
> the
> >>>> compressdev struct fields with the mbuf meta-data, using rte_pktmbuf_data_len(),
> >>>> rte_pktmbuf_mtod(), rte_pktmbuf_mtophys(), etc
> >>>> For simplicity I'd prefer to offer only 1 rather than 2 data formats on the API.
> >>>> We see storage applications rather than IPComp as the main use-case for compressdev, so would
> prefer
> >>>> to optimise for that.
> >>>> Do you think otherwise?
> >>> [Shally] Yea. We plan to use it for ipcomp and other such possible n/w apps. So, we envision mbuf
> support
> >>> as necessary. So, I think we can add two APIs one which process on rte_comp_op and other on
> rte_mbufs
> >>> to make it simpler.
> >>>
> >>>>> Further comments, on github.
> >>>>>
> >>>>> Thanks
> >>>>> Shally
> >>>>>
> >>>>>> -----Original Message-----
> >>>>>> From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> >>>>>> Sent: 12 March 2018 21:31
> >>>>>> To: Ahmed Mansour <ahmed.mansour at nxp.com>; Verma, Shally <Shally.Verma at cavium.com>;
> >>>>> dev at dpdk.org
> >>>>>> Cc: De Lara Guarch, Pablo <pablo.de.lara.guarch at intel.com>; Athreya, Narayana Prasad
> >>>>> <NarayanaPrasad.Athreya at cavium.com>;
> >>>>>> Gupta, Ashish <Ashish.Gupta at cavium.com>; Sahu, Sunila <Sunila.Sahu at cavium.com>; Challa,
> >>> Mahipal
> >>>>>> <Mahipal.Challa at cavium.com>; Jain, Deepak K <deepak.k.jain at intel.com>; Hemant Agrawal
> >>>>> <hemant.agrawal at nxp.com>; Roy
> >>>>>> Pledge <roy.pledge at nxp.com>; Youri Querry <youri.querry_1 at nxp.com>; fiona.trahe at gmail.com;
> >>> Daly,
> >>>>> Lee <lee.daly at intel.com>;
> >>>>>> Jozwiak, TomaszX <tomaszx.jozwiak at intel.com>
> >>>>>> Subject: RE: [dpdk-dev] [PATCH] compressdev: implement API - mbuf alternative
> >>>>>>
> >>>>>> Hi Shally, Ahmed, and anyone else interested in compressdev,
> >>>>>>
> >>>>>> I mentioned last week that we've been exploring using something other than mbufs to pass src/dst
> >>>>> buffers to compressdev PMDs.
> >>>>>> Reasons:
> >>>>>> - mbuf data is limited to 64k-1 in each segment of a chained mbuf. Data for compression
> >>>>>>    can be greater and it would add cycles to have to break up into smaller segments.
> >>>>>> - data may originate in mbufs, but is more likely, particularly for storage use-cases,  to
> >>>>>>    originate in other data structures.
> >>>>>> - There's a 2 cache-line overhead for every segment in a chain, most of this data
> >>>>>>    is network-related, not needed by compressdev
> >>>>>> So moving to a custom structure would minimise memory overhead, remove restriction on 64k-1
> size
> >>> and
> >>>>> give more flexibility if
> >>>>>> compressdev ever needs any comp-specific meta-data.
> >>>>>>
> >>>>>> We've come up with a compressdev-specific structure using the struct iovec from sys/uio.h, which
> is
> >>>>> commonly used by storage
> >>>>>> applications. This would replace the src and dest mbufs in the  op.
> >>>>>> I'll not include the code here - Pablo will push that to github shortly and we'd appreciate review
> >>>>> comments there.
> >>>>>>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpablodelara%2Fd
> pdk-draft-
> >compressdev&data=02%7C01%7Cahmed.mansour%40nxp.com%7C6a8977f9b3714d58621708d589dae
> 567%7C686ea1d3bc2b4c6fa92cd
> >99c5c301635%7C0%7C0%7C636566495639618830&sdata=wmFrxeUNyXdxI5%2Fp5gCmyIRfeDnbHebBJ
> XbztqdsMrc%3D&reserved=0
> >>>>>> Just posting on the mailing list to give a heads-up and ensure this reaches a wider audience than
> may
> >>> see
> >>>>> it on github.
> >>>>>> Note : We also considered having no data structures in the op, instead the application
> >>>>>> would supply a callback which the PMD would use to retrieve meta-data (virt address, iova, length)
> >>>>>> for each next segment as needed. While this is quite flexible and allow the application
> >>>>>> to keep its data in its native structures, it's likely to cost more cycles.
> >>>>>> So we're not proposing this at the moment, but hope to benchmark it later while the API is still
> >>>>> experimental.
> >>>>>> General feedback on direction is welcome here on the mailing list.
> >>>>>> For feedback on the details of implementation we would appreciate comments on github.
> >>>>>>
> >>>>>> Regards,
> >>>>>> Fiona.
> >



More information about the dev mailing list