[dpdk-dev] mbuf next field belongs in the first cacheline

[Morten Brørup]

> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Bruce Richardson
> Sent: Tuesday, 15 June 2021 15.05
> 
> On Tue, Jun 15, 2021 at 02:16:27PM +0200, Morten Brørup wrote:
> > MBUF and MLX5 maintainers,
> >
> > I'm picking up an old discussion, which you might consider pursuing.
> Feel free to ignore, if you consider this discussion irrelevant or
> already closed and done with.
> >
> > The Techboard has previously discussed the organization of the mbuf
> fields. Ref: http://mails.dpdk.org/archives/dev/2020-
> November/191859.html
> >
> > It was concluded that there was no measured performance difference if
> the "pool" or "next" field was in the first cacheline, so it was
> decided to put the "pool" field in the first cacheline. And further
> optimizing the mbuf field organization could be reconsidered later.
> >
> > I have been looking at it. In theory it should not be required to
> touch the "pool" field at RX. But the "next" field must be written for
> segmented packets.
> >
> Question: are there cases where segmented packets are used, but they
> aren't
> big packets, and so need a high packets-per-second value? The thinking
> when
> designing the mbuf was that any application which could handle high
> packets
> per second for medium/small packets would be fine with a few extra
> cycles
> penalty for big ones, since the overall PPS for the driver would be
> much
> lower.

Always good with a reality check! :-)

I recall a proposal from NVIDIA that introduced a feature to split RX packets into multiple small segments from a list of mbuf pools; basically a variant of "header split". Here it is: https://patchwork.dpdk.org/project/dpdk/list/?series=13070&state=%2A&archive=both

I don't know if swapping "next" and "pool" fields would make a performance difference if the RTE_ETH_RX_OFFLOAD_BUFFER_SPLIT is being used.

Otherwise, you are correct: The performance gain is mostly theoretical.

So in reality, it would be a very big change for an insignificant improvement.

It's mainly the principle that annoys me: The DPDK documentation mentions that the mbuf structure is designed for the second cache line not to be touched by RX. If that is not the bearing principle anymore, the documentation needs to be updated.