[dpdk-dev] dmadev discussion summary

[Morten Brørup]

> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of fengchengwen
> Sent: Saturday, 3 July 2021 11.45
> 
> On 2021/7/3 16:53, Morten Brørup wrote:
> >> From: fengchengwen [mailto:fengchengwen at huawei.com]
> >> Sent: Saturday, 3 July 2021 02.32
> >>
> >> On 2021/7/2 22:57, Morten Brørup wrote:
> >>>> In the DPDK framework, many data-plane API names contain queues.
> >> e.g.
> >>>> eventdev/crypto..
> >>>> The concept of virt queues has continuity.
> >>>
> >>> I was also wondering about the name "virtual queue".
> >>>
> >>> Usually, something "virtual" would be an abstraction of something
> >> physical, e.g. a software layer on top of something physical.
> >>>
> >>> Back in the days, a "DMA channel" used to mean a DMA engine on a
> CPU.
> >> If a CPU had 2 DMA channels, they could both be set up
> simultaneously.
> >>>
> >>> The current design has the "dmadev" representing a CPU or other
> chip,
> >> which has one or more "HW-queues" representing DMA channels (of the
> >> same type), and then "virt-queue" as a software abstraction on top,
> for
> >> using a DMA channel in different ways through individually
> configured
> >> contexts (virt-queues).
> >>>
> >>> It makes sense to me, although I would consider renaming "HW-queue"
> >> to "channel" and perhaps "virt-queue" to "queue".
> >>
> >> The 'DMA channel' is more used than 'DMA queue', at least google
> show
> >> that there are at least 20+ times more.
> >>
> >> It's a good idea build the abstraction layer: queue <> channel <>
> dma-
> >> controller.
> >> In this way, the meaning of each layer is relatively easy to
> >> distinguish literally.
> >>
> >> will fix in V2
> >>
> >
> > After re-reading all the mails in this thread, I have found one more
> important high level detail still not decided:
> >
> > Bruce had suggested flattening the DMA channels, so each dmadev
> represents a DMA channel. And DMA controllers with multiple DMA
> channels will have to instantiate multiple dmadevs, one for each DMA
> channel.
> 
> The dpaa2_qdma support multiple DMA channels, I looked into the
> dpaa2_qdma
> and found the control-plane interacts with the kernel, so if we use the
> flattening model, there maybe interactions between dmadevs.

It is perfectly acceptable for the control-plane DMA controller functions to interact across multiple dmadevs, not being thread safe and using locks etc. to protect critical regions accessing shared registers.

The key question is: Can the data-plane dmadev functions (rte_dma_copy() etc.) be implemented to be thread safe, so multiple threads can use data-plane dmadev functions simultaneously?

> 
> >
> > Just like a four port NIC instantiates four ethdevs.
> >
> > Then, like ethdevs, there would only be two abstraction layers:
> dmadev <> queue, where a dmadev is a DMA channel on a DMA controller.
> 
> the dmadev <> channel <> queue model, there are three abstraction
> layers,
> and two abstraction layers.
> 
> >
> > However, this assumes that the fast path functions on the individual
> DMA channels of a DMA controller can be accessed completely
> independently and simultaneously by multiple threads. (Otherwise, the
> driver would need to implement critical regions or locking around
> accessing the common registers in the DMA controller shared by the DMA
> channels.)
> 
> Yes, this scheme has a big implicit dependency, that is, the channels
> are
> independent of each other.
> 
> >
> > Unless any of the DMA controller vendors claim that this assumption
> about independence of the DMA channels is wrong, I strongly support
> Bruce's flattening suggestion.
> >
> > -Morten
> >
>