[dpdk-dev] [PATCH v2 1/8] eal: introduce DMA memory barriers

[Yongseok Koh]

> On Jan 17, 2018, at 5:46 AM, Thomas Monjalon <thomas at monjalon.net> wrote:
> 
> 16/01/2018 10:10, Jianbo Liu:
>> The 01/16/2018 10:49, Andrew Rybchenko wrote:
>>> On 01/16/2018 04:10 AM, Yongseok Koh wrote:
>>>> This commit introduces rte_dma_wmb() and rte_dma_rmb(), in order to
>>>> guarantee the ordering of coherent shared memory between the CPU and a DMA
>>>> capable device.
>>>> 
>>>> Signed-off-by: Yongseok Koh <yskoh at mellanox.com>
>>>> ---
>>>> lib/librte_eal/common/include/generic/rte_atomic.h | 18 ++++++++++++++++++
>>>> 1 file changed, 18 insertions(+)
>>>> 
>>>> diff --git a/lib/librte_eal/common/include/generic/rte_atomic.h b/lib/librte_eal/common/include/generic/rte_atomic.h
>>>> index 16af5ca57..2e0503ce6 100644
>>>> --- a/lib/librte_eal/common/include/generic/rte_atomic.h
>>>> +++ b/lib/librte_eal/common/include/generic/rte_atomic.h
>>>> @@ -98,6 +98,24 @@ static inline void rte_io_wmb(void);
>>>>  */
>>>> static inline void rte_io_rmb(void);
>>>> +/**
>>>> + * Write memory barrier for coherent memory between lcore and IO device
>>>> + *
>>>> + * Guarantees that the STORE operations on coherent memory that
>>>> + * precede the rte_dma_wmb() call are visible to I/O device before the
>>>> + * STORE operations that follow it.
>>>> + */
>>>> +static inline void rte_dma_wmb(void);
>>>> +
>>>> +/**
>>>> + * Read memory barrier for coherent memory between lcore and IO device
>>>> + *
>>>> + * Guarantees that the LOAD operations on coherent memory updated by
>>>> + * IO device that precede the rte_dma_rmb() call are visible to CPU
>>>> + * before the LOAD operations that follow it.
>>>> + */
>>>> +static inline void rte_dma_rmb(void);
>>>> +
>>>> #endif /* __DOXYGEN__ */
>>>> /**
>>> 
>>> I'm not an ARMv8 expert so, my comments could be a bit ignorant.
>>> I'd like to understand the difference between io and added here dma
>>> barriers.
>>> The difference should be clearly explained. Otherwise we'll constantly hit
>>> on incorrect choice of barrier type.
>>> Also I don't understand why "dma" name is chosen taking into account
>>> that definition is bound to coherent memory between lcore and IO device.
>> 
>> A good explanation can be found here.
>> 
>> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Fcommit%2F%3Fid%3D1077fa36f23e259858caf6f269a47393a5aff523&data=02%7C01%7Cyskoh%40mellanox.com%7C7b526265cbf1449f3db208d55db0c55d%7Ca652971c7d2e4d9ba6a4d149256f461b%7C0%7C0%7C636517936183877836&sdata=2%2Fi8Gs2n%2Fnbe9%2FJ3GWr22ndPmQVmvM2Xh12r3j1ZWlg%3D&reserved=0
> 
> I agree that something more is needed to explain when to use rte_io_*.
> The only difference between rte_io_* and rte_dma_* is "on coherent memory".

Okay will add more explanation and send out v3 soon. But, please note that
there's no concrete theory when to use which barrier. Actually, it is mostly
for ARMv8 because it provides more options for barriers. For other archs, as you
can see in the patches, there's no difference from IO barriers.

Thanks,
Yongseok