[PATCH v14 0/6] introduce memarea library

[fengchengwen]

Hi Ferruh,

On 2023/6/12 21:53, Ferruh Yigit wrote:
> On 2/9/2023 6:36 AM, Chengwen Feng wrote:
>> The memarea library is an allocator of variable-size object which based
>> on a memory region. The main features are as follows:
>>
>> - The memory region can be initialized from the following memory
>>   sources:
>>   1. HEAP: e.g. invoke rte_malloc_socket.
>>   2. LIBC: e.g. invoke posix_memalign.
>>   3. Another memarea: it can be from another memarea.
>>
>> - It supports MT-safe as long as it's specified at creation time.
>>
>> Note:
>> a) The memarea is oriented towards the application layer, which could
>> provides 'region-based memory management' [1] function.
>> b) The eal library also provide memory zone/heap management, but these
>> are tied to huge pages management.
>>
>> [1] https://en.wikipedia.org/wiki/Region-based_memory_management
>>
>> Chengwen Feng (6):
>>   memarea: introduce memarea library
>>   test/memarea: support memarea test
>>   memarea: support alloc and free API
>>   test/memarea: support alloc and free API test
>>   memarea: support dump API
>>   test/memarea: support dump API test
>>
> 
> Hi Chengwen,
> 
> I am aware that patchset it out for a while now, and library has a good
> feature set, but I am curious what problem this library solves that
> existing APIs can't.
> Can you please describe more what is the usecase and motivation of the
> feature/library?

Please see the reply below.

> 
> 
> I can see it is easy/cheap to destroy a memory region, so is your
> problem memory free performance?
> Mostly mempool is used to solve this issue, are you planning to use
> these APIs on datapath for variable size buffers?

Yes, it could use on datapath.

Now DPDK is accelerating more areas (not limited to pure packet processing),
The applications have variable size buffers malloc/free requirement on datapath
(e.g. video app decode/encode).

One option is to use rte_malloc, but the rte_malloc is not target for datapath
because it's implementation relies on a global lock.

The optimization direction:
1) optimize rte_malloc API, use something like core-cache tech.
2) change the mindset, make each application session use its own memory region
   (as this library provides), e.g. if have three video decocder run on dpdk
   thread, application could create three memarea for each.
3) hybrid management[1], which use both 1&2.

> 
> 
> Or is the motivation to be able to run memory management on top of
> malloc() allocated memory, if so why?
> Hugepages are used for performance reason, if allocated memory is used
> for datapath, won't this cause performance degradation.

Hugepages is a dessert, it need specific API to management (unless the transparent huge page).

Why provide the memarea over malloc because: the memarea is programmer friendly, we could
alloc one region, and do many alloc subblock, at last free the region.

The application decide whether to use such feature (memarea over malloc).

> And if this is the main motivation, can it be an option to add regular
> page support to memsegments, this way existing APIs can be used with a
> flag, to make it easier for users.

Also, I do some investigation in DPDK recently and found the mlx5 use mlx5_ipool to management
its device memory, I think the memarea could be extended to support such usecase.

So I think the memarea library could use both control and datapath, and also for device memory
management.

[1] https://www.youtube.com/watch?v=YghmGLyqrVg

Thanks.

> 
> 
> Thanks,
> Ferruh
> 
> 
>> ---
>> v14:
>> * address Stephen's comments: RTE_MEMAREA_LOG use easy impl and add
>>   __func__ print.
>> v13:
>> * address Morten's comments: make debug cookies optional, controlled
>>   by RTE_LIBRTE_MEMAREA_DEBUG; disabled by default.
>> * reduce management data overhead.
>> v12:
>> * remove rte_memarea_refcnt_update() API which address Dongdong's
>>   comments.
>> * refine the variable naming.
>> * fix some bugs.
>> v11:
>> * rebase 23.03
>> * remove "app/test: add memarea to malloc-perf-autotest" because the
>>   two algorithm are not comparable which also address previous
>>   comments.
>> v10:
>> * support windows platform.
>> * add rte_memarea.libc perftest to malloc-perf-autotest.
>> v9:
>> * address Dmitry's comments.
>> * drop features of SOURCE_USER and backup memarea mechanism.
>> * rename rte_memarea_update_refcnt to rte_memarea_refcnt_update
>>   to keep with rte_mbuf_refcnt_update name style.
>> * fix memarea perftest compile failed at windows platform.
>> * fix spell warning.
>> v8:
>> * address Mattias's comments (rename ALG_DEFAULT with ALG_NEXTFIT).
>> * small feature patches are combined.
>> * enhanced backup memory mechanism.
>> * add memarea to malloc-perf-autotest.
>> * other tiny naming optimize.
>> v7:
>> * repost patches as there are spread over different series in patchwork.
>> v6:
>> * address Mattias's comments.
>> v5:
>> * fix 09/10 patch spell warning.
>> v4:
>> * repost patches as there are spread over different series in patchwork.
>> v3:
>> * add memory source of RTE memory.
>> * add algorithm field to facilitate the introduction of new algorithms.
>> * fix memarea log don't output problem.
>> v2:
>> * fix compile issues reported by dpdk-test-report.
>> * address Dimitry and Jerin's comments.
>> * add no MT-safe test.
>>
>>  MAINTAINERS                            |   6 +
>>  app/test/meson.build                   |   2 +
>>  app/test/test_memarea.c                | 297 ++++++++++++++++
>>  doc/api/doxy-api-index.md              |   3 +-
>>  doc/api/doxy-api.conf.in               |   1 +
>>  doc/guides/prog_guide/index.rst        |   1 +
>>  doc/guides/prog_guide/memarea_lib.rst  |  55 +++
>>  doc/guides/rel_notes/release_23_03.rst |   6 +
>>  lib/memarea/memarea_private.h          |  58 ++++
>>  lib/memarea/meson.build                |  10 +
>>  lib/memarea/rte_memarea.c              | 448 +++++++++++++++++++++++++
>>  lib/memarea/rte_memarea.h              | 185 ++++++++++
>>  lib/memarea/version.map                |  15 +
>>  lib/meson.build                        |   1 +
>>  14 files changed, 1087 insertions(+), 1 deletion(-)
>>  create mode 100644 app/test/test_memarea.c
>>  create mode 100644 doc/guides/prog_guide/memarea_lib.rst
>>  create mode 100644 lib/memarea/memarea_private.h
>>  create mode 100644 lib/memarea/meson.build
>>  create mode 100644 lib/memarea/rte_memarea.c
>>  create mode 100644 lib/memarea/rte_memarea.h
>>  create mode 100644 lib/memarea/version.map
>>
> 
> .
>