[dpdk-dev] [RFC 0/8] mbuf: structure reorganization

[Ananyev, Konstantin]

> -----Original Message-----
> From: jblunck at gmail.com [mailto:jblunck at gmail.com] On Behalf Of Jan Blunck
> Sent: Tuesday, February 21, 2017 7:18 PM
> To: Ananyev, Konstantin <konstantin.ananyev at intel.com>
> Cc: Richardson, Bruce <bruce.richardson at intel.com>; Olivier MATZ <olivier.matz at 6wind.com>; dev at dpdk.org
> Subject: Re: [dpdk-dev] [RFC 0/8] mbuf: structure reorganization
> 
> On Tue, Feb 21, 2017 at 6:26 PM, Ananyev, Konstantin
> <konstantin.ananyev at intel.com> wrote:
> > Hi Jan,
> >
> >> -----Original Message-----
> >> From: jblunck at gmail.com [mailto:jblunck at gmail.com] On Behalf Of Jan Blunck
> >> Sent: Tuesday, February 21, 2017 5:05 PM
> >> To: Richardson, Bruce <bruce.richardson at intel.com>
> >> Cc: Olivier MATZ <olivier.matz at 6wind.com>; Ananyev, Konstantin <konstantin.ananyev at intel.com>; dev at dpdk.org
> >> Subject: Re: [dpdk-dev] [RFC 0/8] mbuf: structure reorganization
> >>
> >> On Tue, Feb 21, 2017 at 5:38 PM, Bruce Richardson
> >> <bruce.richardson at intel.com> wrote:
> >> > On Tue, Feb 21, 2017 at 05:12:12PM +0100, Jan Blunck wrote:
> >> >>
> >> >> Access through PMD specific function pointers should be relatively
> >> >> fast on access. Modern architecture optimize that use case well
> >> >> enough.
> >> >>
> >> > The cost of doing a function call per packet to access data starts to
> >> > add up very, very fast. For the app, once the data is written to the
> >> > mbuf, it should be in the L1 cache, giving very fast access to it in a
> >> > few cycles. However, if a function call has to be made in order to do
> >> > the read, that makes the read of that field many times more expensive.
> >> >
> >>
> >> Exactly. Right now the timestamp normalization is done before writing
> >> to each mbuf. Timestamps are usually read at most once ... if at all.
> >
> > Well we don't know for sure right?
> > Someone can argue that there are plenty of scenarios  when
> > other fields might also never be used/updated (rss, vlan, etc).
> >
> > So, are you suggesting to do normalization later?
> > If so, then what would be the benefit (data still need to be in mbuf)?
> 
> Yes, postponing normalization prevents you from doing unnecessary work
> upfront. AFAIK not all NICs store timestamp data OOB, e.g. in CQ.

Yes, postponing normalization might help a bit (though I don't think much)
in terms of calculations performed inside PMD.
But we still need 8B inside mbuf to store the timestamp value,
either normalized or raw one.
So to clarify where is the disagreement:
1. timestamp position:
    mbufs 1-st cacheline vs 2-nd cacheline
2. timestamp normalization point
     inside PMD RX vs somewhere later as user needs it (extra function in dev_ops?).

So, is it 1) or 2) or both?         
Konstantin



> 
> >
> >> If you look at the analysis use cases they are read to be written to
> >> persistent storage.
> >
> > Probably, or some statistic calculations, I guess.
> > Or might be someone would use it to reorder packets before sending
> > them out based on the timestamp, or might be something else.
> 
> Those timestamps are generated at arrival in the hardware. So that
> order would be relative to the point of reception and in case you want
> to resync flows that doesn't help much.
> 
> For statistics calculation and the resync use case you also might want
> the raw timestamp (not normalized to nsec) and offload the
> normalization out of band to reduce overall latency.
> 
> > It really hard to predict what use cases would come up (at least for me).
> >
> 
> Indeed. I also just have the limited view of what I used timestamps
> for so far. Who knows what use cases other people come up with in the
> future.