MLX5 PMD access ring library private data

[Konstantin Ananyev]

> 
> On 2023/8/17 22:06, Stephen Hemminger wrote:
> > On Thu, 17 Aug 2023 05:06:20 +0000
> > Honnappa Nagarahalli <Honnappa.Nagarahalli at arm.com> wrote:
> >
> >> Hi Matan, Viacheslav,
> >> 	Tyler pointed out that the function __mlx5_hws_cnt_pool_enqueue_revert is accessing the ring private structure members
> (prod.head and prod.tail) directly. Even though ' struct rte_ring' is a public structure (mainly because the library provides inline
> functions), the structure members are considered private to the ring library. So, this needs to be corrected.
> >>
> >> It looks like the function __mlx5_hws_cnt_pool_enqueue_revert is trying to revert things that were enqueued. It is not clear to
> me why this functionality is required. Can you provide the use case for this? We can discuss possible solutions.
> > How can reverting be thread safe? Consumer could have already looked at them?
> 
> Hey,
> 
> In our case, this ring is SC/SP, only accessed by one thread
> (enqueue/dequeue/revert).
> 
> The scenario we have "revert" is:
> 
>   We use ring to manager our HW objects (counter in this case) and for
> each core (thread) has "cache" (a SC/SP ring) for sake of performance.
> 
> 1. Get objects from "cache" firstly, if cache is empty, we fetch a bulk
> of free objects from global ring into cache.
> 
> 2. Put (free) objects also into "cache" firstly, if cache is full, we
> flush a bulk of objects into global ring in order to make some rooms in
> cache.
> 
> However, this HW object cannot be immediately reused after free. It
> needs time to be reset and then can be used again.
> 
> So when we flush cache, we want to keep the first enqueued objects still
> stay there because they have more chance already be reset than the
> latest enqueued objects.
> 
> Only flush recently enqueued objects back into global ring, act as
> "LIFO" behavior.
> 
> This is why we require "revert" enqueued objects.
> 

Wouldn't then simple stack fit you better?
Something like lib/stack/rte_stack_std.h, but even without spinlock around?