[dpdk-dev] [RFC] Compression API in DPDK

[Verma, Shally]

Ok. Assume that will be self-contained unlike current which has references to crypto?!

Thanks
Shally

> -----Original Message-----
> From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> Sent: 20 November 2017 17:04
> To: Verma, Shally <Shally.Verma at cavium.com>; dev at dpdk.org
> Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> Challa, Mahipal <Mahipal.Challa at cavium.com>; Trahe, Fiona
> <fiona.trahe at intel.com>
> Subject: RE: [dpdk-dev] [RFC] Compression API in DPDK
> 
> Hi Shally,
> I'm aiming to get a v2 of the RFC out this week.
> Regards,
> Fiona
> 
> > -----Original Message-----
> > From: Verma, Shally [mailto:Shally.Verma at cavium.com]
> > Sent: Monday, November 20, 2017 6:44 AM
> > To: Verma, Shally <Shally.Verma at cavium.com>; Trahe, Fiona
> <fiona.trahe at intel.com>; dev at dpdk.org
> > Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> Challa, Mahipal
> > <Mahipal.Challa at cavium.com>
> > Subject: RE: [dpdk-dev] [RFC] Compression API in DPDK
> >
> > Hi Fiona
> >
> > Could you give some expected timeframe for next comp API spec patch?
> >
> > Thanks
> > Shally
> >
> > > -----Original Message-----
> > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Verma, Shally
> > > Sent: 10 November 2017 17:35
> > > To: Trahe, Fiona <fiona.trahe at intel.com>; dev at dpdk.org
> > > Cc: Athreya, Narayana Prasad <NarayanaPrasad.Athreya at cavium.com>;
> > > Challa, Mahipal <Mahipal.Challa at cavium.com>
> > > Subject: Re: [dpdk-dev] [RFC] Compression API in DPDK
> > >
> > > [This sender failed our fraud detection checks and may not be who they
> > > appear to be. Learn about spoofing at
> http://aka.ms/LearnAboutSpoofing]
> > >
> > > > -----Original Message-----
> > > > From: Trahe, Fiona [mailto:fiona.trahe at intel.com]
> > > > Sent: 07 November 2017 16:54
> > > > To: Verma, Shally <Shally.Verma at cavium.com>; dev at dpdk.org
> > > > Cc: Athreya, Narayana Prasad
> <NarayanaPrasad.Athreya at cavium.com>;
> > > > Challa, Mahipal <Mahipal.Challa at cavium.com>; Trahe, Fiona
> > > > <fiona.trahe at intel.com>
> > > > Subject: RE: [dpdk-dev] [RFC] Compression API in DPDK
> > > >
> > > > Hi Shally,
> > > >
> > > > ///snip///
> > > > > [Shally] Ok. Then, just to confirm my understanding here. You mean
> PMD
> > > > can figure out amount of
> > > > > available space in dst mbuf by calling rte_pktmbuf_data_len() on each
> of
> > > its
> > > > segment?
> > > > [Fiona] exactly.
> > > >
> > > > ///snip///
> > > > > > > > > > +      * This indicates the buffer size and should be
> > > > > > > > > > +      * set a little larger than the expected max source buffer
> > > size.
> > > > > > > > > > +      * if the output of static compression doesn't fit in the
> > > > > > > > > > +      * intermediate buffer dynamic compression may not be
> > > > possible,
> > > > > > > > > > +      * in this case the accelerator may revert back to static
> > > > > > compression.
> > > > > [Shally] > > > > +      * in this case the accelerator may revert back to
> static
> > > > compression.> > > > > +      */
> > > > > Can you elaborate more on this? This looks to me as decision made
> during
> > > > enqueue_burst() processing.
> > > > > If yes and If application has chosen specific Huffman code i.e.
> > > > RTE_COMP_DYNAMIC or
> > > > > RTE_COMP_FIXED in rte_comp_compress_xform, then how this
> would
> > > > work?
> > > > [Fiona] yes, it would have to revert back on the enqueue. The
> compressed
> > > > data would still conform to deflate standard, so any decompressor
> would
> > > be
> > > > able to inflate it. The ratio would not be as good as hoped for but it
> would
> > > be
> > > > the best the compression engine could do with the resources it has.
> > > >
> > > [Shally] Ok. However, I'm not sure how to use Intermediate bufs here as
> it is
> > > not requirement for us for this purpose.
> > > So, it looks like It is very device specific requirement where some may not
> > > need it. So, I would suggest that API should propose a way to indicate if
> it's a
> > > requirement for specific device so that app can input it at config time.
> May be
> > > feature flag or capability.
> > >
> > > Thanks
> > > Shally
> > >
> > > > ///snip///
> > > > > [Shally] Sure. So just to align here. Except few questions posted
> above on
> > > > this RFC (such as Dynamic Vs
> > > > > Static or dst mbuf parsing), following (and any other) will further be
> > > > covered as part of 'RFC doc'
> > > > > discussion
> > > > > - Hash support
> > > > > - RTE_COMPDEV_FF_MULTI_PKT_CHECKSUM
> > > > [Fiona] Agreed.