[v2] mbuf: support dynamic fields and flags
Checks
Commit Message
Many features require to store data inside the mbuf. As the room in mbuf
structure is limited, it is not possible to have a field for each
feature. Also, changing fields in the mbuf structure can break the API
or ABI.
This commit addresses these issues, by enabling the dynamic registration
of fields or flags:
- a dynamic field is a named area in the rte_mbuf structure, with a
given size (>= 1 byte) and alignment constraint.
- a dynamic flag is a named bit in the rte_mbuf structure.
The typical use case is a PMD that registers space for an offload
feature, when the application requests to enable this feature. As
the space in mbuf is limited, the space should only be reserved if it
is going to be used (i.e when the application explicitly asks for it).
The registration can be done at any moment, but it is not possible
to unregister fields or flags for now.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
---
v2
* Rebase on top of master: solve conflict with Stephen's patchset
(packet copy)
* Add new apis to register a dynamic field/flag at a specific place
* Add a dump function (sugg by David)
* Enhance field registration function to select the best offset, keeping
large aligned zones as much as possible (sugg by Konstantin)
* Use a size_t and unsigned int instead of int when relevant
(sugg by Konstantin)
* Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
(sugg by Konstantin)
* Remove unused argument in private function (sugg by Konstantin)
* Fix and simplify locking (sugg by Konstantin)
* Fix minor typo
rfc -> v1
* Rebase on top of master
* Change registration API to use a structure instead of
variables, getting rid of #defines (Stephen's comment)
* Update flag registration to use a similar API as fields.
* Change max name length from 32 to 64 (sugg. by Thomas)
* Enhance API documentation (Haiyue's and Andrew's comments)
* Add a debug log at registration
* Add some words in release note
* Did some performance tests (sugg. by Andrew):
On my platform, reading a dynamic field takes ~3 cycles more
than a static field, and ~2 cycles more for writing.
app/test/test_mbuf.c | 145 ++++++-
doc/guides/rel_notes/release_19_11.rst | 7 +
lib/librte_mbuf/Makefile | 2 +
lib/librte_mbuf/meson.build | 6 +-
lib/librte_mbuf/rte_mbuf.h | 23 +-
lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
lib/librte_mbuf/rte_mbuf_version.map | 7 +
8 files changed, 959 insertions(+), 5 deletions(-)
create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
Comments
Hi Olivier
> -----Original Message-----
> From: Olivier Matz [mailto:olivier.matz@6wind.com]
> Sent: Thursday, October 17, 2019 22:42
> To: dev@dpdk.org
> Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>; Wang,
> Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: [PATCH v2] mbuf: support dynamic fields and flags
>
> Many features require to store data inside the mbuf. As the room in mbuf
> structure is limited, it is not possible to have a field for each
> feature. Also, changing fields in the mbuf structure can break the API
> or ABI.
>
> This commit addresses these issues, by enabling the dynamic registration
> of fields or flags:
>
> - a dynamic field is a named area in the rte_mbuf structure, with a
> given size (>= 1 byte) and alignment constraint.
> - a dynamic flag is a named bit in the rte_mbuf structure.
>
> The typical use case is a PMD that registers space for an offload
> feature, when the application requests to enable this feature. As
> the space in mbuf is limited, the space should only be reserved if it
> is going to be used (i.e when the application explicitly asks for it).
>
> The registration can be done at any moment, but it is not possible
> to unregister fields or flags for now.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>
> v2
>
> * Rebase on top of master: solve conflict with Stephen's patchset
> (packet copy)
> * Add new apis to register a dynamic field/flag at a specific place
> * Add a dump function (sugg by David)
> * Enhance field registration function to select the best offset, keeping
> large aligned zones as much as possible (sugg by Konstantin)
> * Use a size_t and unsigned int instead of int when relevant
> (sugg by Konstantin)
> * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> (sugg by Konstantin)
> * Remove unused argument in private function (sugg by Konstantin)
> * Fix and simplify locking (sugg by Konstantin)
> * Fix minor typo
>
> rfc -> v1
>
> * Rebase on top of master
> * Change registration API to use a structure instead of
> variables, getting rid of #defines (Stephen's comment)
> * Update flag registration to use a similar API as fields.
> * Change max name length from 32 to 64 (sugg. by Thomas)
> * Enhance API documentation (Haiyue's and Andrew's comments)
> * Add a debug log at registration
> * Add some words in release note
> * Did some performance tests (sugg. by Andrew):
> On my platform, reading a dynamic field takes ~3 cycles more
> than a static field, and ~2 cycles more for writing.
>
> app/test/test_mbuf.c | 145 ++++++-
> doc/guides/rel_notes/release_19_11.rst | 7 +
> lib/librte_mbuf/Makefile | 2 +
> lib/librte_mbuf/meson.build | 6 +-
> lib/librte_mbuf/rte_mbuf.h | 23 +-
> lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> lib/librte_mbuf/rte_mbuf_version.map | 7 +
> 8 files changed, 959 insertions(+), 5 deletions(-)
> create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
>
> diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> index b9c2b2500..01cafad59 100644
> --- a/app/test/test_mbuf.c
> +++ b/app/test/test_mbuf.c
> @@ -28,6 +28,7 @@
> #include <rte_random.h>
[snip]
> +/**
> + * Helper macro to access to a dynamic field.
> + */
> +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) + (offset)))
> +
The suggested macro is missed ? ;-)
/**
* Helper macro to access to a dynamic flag.
*/
#define RTE_MBUF_DYNFLAG(offset) (1ULL << (offset))
BTW, should we have a place to put the registered dynamic fields and flags
names together (a name overview -- detail Link to --> PMD's help page) ?
Since rte_mbuf_dynfield:name & rte_mbuf_dynflag:name work as a API style,
users can check how many 'names' registered, developers can check whether
the names they want to use are registered or not ? They don't need to have
to check the rte_errno ... Just a suggestion for user experience.
>
> } DPDK_18.08;
> --
> 2.20.1
Hi Haiyue,
On Fri, Oct 18, 2019 at 02:47:50AM +0000, Wang, Haiyue wrote:
> Hi Olivier
>
> > -----Original Message-----
> > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > Sent: Thursday, October 17, 2019 22:42
> > To: dev@dpdk.org
> > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>; Wang,
> > Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> > <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> > <thomas@monjalon.net>
> > Subject: [PATCH v2] mbuf: support dynamic fields and flags
> >
> > Many features require to store data inside the mbuf. As the room in mbuf
> > structure is limited, it is not possible to have a field for each
> > feature. Also, changing fields in the mbuf structure can break the API
> > or ABI.
> >
> > This commit addresses these issues, by enabling the dynamic registration
> > of fields or flags:
> >
> > - a dynamic field is a named area in the rte_mbuf structure, with a
> > given size (>= 1 byte) and alignment constraint.
> > - a dynamic flag is a named bit in the rte_mbuf structure.
> >
> > The typical use case is a PMD that registers space for an offload
> > feature, when the application requests to enable this feature. As
> > the space in mbuf is limited, the space should only be reserved if it
> > is going to be used (i.e when the application explicitly asks for it).
> >
> > The registration can be done at any moment, but it is not possible
> > to unregister fields or flags for now.
> >
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> >
> > v2
> >
> > * Rebase on top of master: solve conflict with Stephen's patchset
> > (packet copy)
> > * Add new apis to register a dynamic field/flag at a specific place
> > * Add a dump function (sugg by David)
> > * Enhance field registration function to select the best offset, keeping
> > large aligned zones as much as possible (sugg by Konstantin)
> > * Use a size_t and unsigned int instead of int when relevant
> > (sugg by Konstantin)
> > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > (sugg by Konstantin)
> > * Remove unused argument in private function (sugg by Konstantin)
> > * Fix and simplify locking (sugg by Konstantin)
> > * Fix minor typo
> >
> > rfc -> v1
> >
> > * Rebase on top of master
> > * Change registration API to use a structure instead of
> > variables, getting rid of #defines (Stephen's comment)
> > * Update flag registration to use a similar API as fields.
> > * Change max name length from 32 to 64 (sugg. by Thomas)
> > * Enhance API documentation (Haiyue's and Andrew's comments)
> > * Add a debug log at registration
> > * Add some words in release note
> > * Did some performance tests (sugg. by Andrew):
> > On my platform, reading a dynamic field takes ~3 cycles more
> > than a static field, and ~2 cycles more for writing.
> >
> > app/test/test_mbuf.c | 145 ++++++-
> > doc/guides/rel_notes/release_19_11.rst | 7 +
> > lib/librte_mbuf/Makefile | 2 +
> > lib/librte_mbuf/meson.build | 6 +-
> > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > 8 files changed, 959 insertions(+), 5 deletions(-)
> > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> >
> > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > index b9c2b2500..01cafad59 100644
> > --- a/app/test/test_mbuf.c
> > +++ b/app/test/test_mbuf.c
> > @@ -28,6 +28,7 @@
> > #include <rte_random.h>
>
> [snip]
>
> > +/**
> > + * Helper macro to access to a dynamic field.
> > + */
> > +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) + (offset)))
> > +
>
> The suggested macro is missed ? ;-)
> /**
> * Helper macro to access to a dynamic flag.
> */
> #define RTE_MBUF_DYNFLAG(offset) (1ULL << (offset))
Yes, sorry.
Thinking a bit more about it, I wonder if the macros below aren't
more consistent with the dynamic field (because they take the mbuf
as parameter)?
#define RTE_MBUF_SET_DYNFLAG(m, bitnum, val) ...
#define RTE_MBUF_GET_DYNFLAG(m, bitnum) ...
They could even be static inline functions.
On the other hand, these helpers would be generic to ol_flags, not only
for dynamic flags. Today, we use (1ULL << bit) for ol_flags, which makes
me wonder... is the macro really needed after all? :)
> BTW, should we have a place to put the registered dynamic fields and flags
> names together (a name overview -- detail Link to --> PMD's help page) ?
The centralized place will be in rte_mbuf_dyn.h for fields/flags that can
are shared between several dpdk areas. Some libraries/pmd could have private
dynamic fields/flags. In any case, I think the same namespace than functions
should be used. Probably something like this:
- "rte_mbuf_dynfield_<name>" in mbuf lib
- "rte_<libname>_dynfield_<name>" in other libs
- "rte_net_<pmd>_dynfield_<name>" in pmds
- "<name>" in apps
> Since rte_mbuf_dynfield:name & rte_mbuf_dynflag:name work as a API style,
> users can check how many 'names' registered, developers can check whether
> the names they want to use are registered or not ? They don't need to have
> to check the rte_errno ... Just a suggestion for user experience.
I did not get you point. Does my response above answers to your question?
Regards,
Olivier
Hi Olivier,
> -----Original Message-----
> From: Olivier Matz [mailto:olivier.matz@6wind.com]
> Sent: Friday, October 18, 2019 15:54
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org; Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: Re: [PATCH v2] mbuf: support dynamic fields and flags
>
> Hi Haiyue,
>
> On Fri, Oct 18, 2019 at 02:47:50AM +0000, Wang, Haiyue wrote:
> > Hi Olivier
> >
> > > -----Original Message-----
> > > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > > Sent: Thursday, October 17, 2019 22:42
> > > To: dev@dpdk.org
> > > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Wang,
> > > Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > > <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> > > <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> > > <thomas@monjalon.net>
> > > Subject: [PATCH v2] mbuf: support dynamic fields and flags
> > >
> > > Many features require to store data inside the mbuf. As the room in mbuf
> > > structure is limited, it is not possible to have a field for each
> > > feature. Also, changing fields in the mbuf structure can break the API
> > > or ABI.
> > >
> > > This commit addresses these issues, by enabling the dynamic registration
> > > of fields or flags:
> > >
> > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > given size (>= 1 byte) and alignment constraint.
> > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > >
> > > The typical use case is a PMD that registers space for an offload
> > > feature, when the application requests to enable this feature. As
> > > the space in mbuf is limited, the space should only be reserved if it
> > > is going to be used (i.e when the application explicitly asks for it).
> > >
> > > The registration can be done at any moment, but it is not possible
> > > to unregister fields or flags for now.
> > >
> > > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > > ---
> > >
> > > v2
> > >
> > > * Rebase on top of master: solve conflict with Stephen's patchset
> > > (packet copy)
> > > * Add new apis to register a dynamic field/flag at a specific place
> > > * Add a dump function (sugg by David)
> > > * Enhance field registration function to select the best offset, keeping
> > > large aligned zones as much as possible (sugg by Konstantin)
> > > * Use a size_t and unsigned int instead of int when relevant
> > > (sugg by Konstantin)
> > > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > > (sugg by Konstantin)
> > > * Remove unused argument in private function (sugg by Konstantin)
> > > * Fix and simplify locking (sugg by Konstantin)
> > > * Fix minor typo
> > >
> > > rfc -> v1
> > >
> > > * Rebase on top of master
> > > * Change registration API to use a structure instead of
> > > variables, getting rid of #defines (Stephen's comment)
> > > * Update flag registration to use a similar API as fields.
> > > * Change max name length from 32 to 64 (sugg. by Thomas)
> > > * Enhance API documentation (Haiyue's and Andrew's comments)
> > > * Add a debug log at registration
> > > * Add some words in release note
> > > * Did some performance tests (sugg. by Andrew):
> > > On my platform, reading a dynamic field takes ~3 cycles more
> > > than a static field, and ~2 cycles more for writing.
> > >
> > > app/test/test_mbuf.c | 145 ++++++-
> > > doc/guides/rel_notes/release_19_11.rst | 7 +
> > > lib/librte_mbuf/Makefile | 2 +
> > > lib/librte_mbuf/meson.build | 6 +-
> > > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > > 8 files changed, 959 insertions(+), 5 deletions(-)
> > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> > >
> > > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > > index b9c2b2500..01cafad59 100644
> > > --- a/app/test/test_mbuf.c
> > > +++ b/app/test/test_mbuf.c
> > > @@ -28,6 +28,7 @@
> > > #include <rte_random.h>
> >
> > [snip]
> >
> > > +/**
> > > + * Helper macro to access to a dynamic field.
> > > + */
> > > +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) + (offset)))
> > > +
> >
> > The suggested macro is missed ? ;-)
> > /**
> > * Helper macro to access to a dynamic flag.
> > */
> > #define RTE_MBUF_DYNFLAG(offset) (1ULL << (offset))
>
> Yes, sorry.
>
> Thinking a bit more about it, I wonder if the macros below aren't
> more consistent with the dynamic field (because they take the mbuf
> as parameter)?
>
> #define RTE_MBUF_SET_DYNFLAG(m, bitnum, val) ...
> #define RTE_MBUF_GET_DYNFLAG(m, bitnum) ...
>
> They could even be static inline functions.
>
> On the other hand, these helpers would be generic to ol_flags, not only
> for dynamic flags. Today, we use (1ULL << bit) for ol_flags, which makes
> me wonder... is the macro really needed after all? :)
>
I used as this:
1). in PMD:
mb->ol_flags |= RTE_MBUF_DYNFLAG(ol_offset);
2). In testpmd
if (mb->ol_flags & RTE_MBUF_DYNFLAG(ol_offset))
...
The above two macros look better in real use.
> > BTW, should we have a place to put the registered dynamic fields and flags
> > names together (a name overview -- detail Link to --> PMD's help page) ?
>
> The centralized place will be in rte_mbuf_dyn.h for fields/flags that can
> are shared between several dpdk areas. Some libraries/pmd could have private
> dynamic fields/flags. In any case, I think the same namespace than functions
> should be used. Probably something like this:
> - "rte_mbuf_dynfield_<name>" in mbuf lib
> - "rte_<libname>_dynfield_<name>" in other libs
> - "rte_net_<pmd>_dynfield_<name>" in pmds
> - "<name>" in apps
>
> > Since rte_mbuf_dynfield:name & rte_mbuf_dynflag:name work as a API style,
> > users can check how many 'names' registered, developers can check whether
> > the names they want to use are registered or not ? They don't need to have
> > to check the rte_errno ... Just a suggestion for user experience.
>
> I did not get you point. Does my response above answers to your question?
>
Yes, the name conversation you mentioned above is a good practice, then no doc
needed any more, thanks!
> Regards,
> Olivier
On Fri, Oct 18, 2019 at 08:28:02AM +0000, Wang, Haiyue wrote:
> Hi Olivier,
>
> > -----Original Message-----
> > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > Sent: Friday, October 18, 2019 15:54
> > To: Wang, Haiyue <haiyue.wang@intel.com>
> > Cc: dev@dpdk.org; Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce
> > <bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> > <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> > <thomas@monjalon.net>
> > Subject: Re: [PATCH v2] mbuf: support dynamic fields and flags
> >
> > Hi Haiyue,
> >
> > On Fri, Oct 18, 2019 at 02:47:50AM +0000, Wang, Haiyue wrote:
> > > Hi Olivier
> > >
> > > > -----Original Message-----
> > > > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > > > Sent: Thursday, October 17, 2019 22:42
> > > > To: dev@dpdk.org
> > > > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> > Wang,
> > > > Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > > > <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> > > > <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> > > > <thomas@monjalon.net>
> > > > Subject: [PATCH v2] mbuf: support dynamic fields and flags
> > > >
> > > > Many features require to store data inside the mbuf. As the room in mbuf
> > > > structure is limited, it is not possible to have a field for each
> > > > feature. Also, changing fields in the mbuf structure can break the API
> > > > or ABI.
> > > >
> > > > This commit addresses these issues, by enabling the dynamic registration
> > > > of fields or flags:
> > > >
> > > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > > given size (>= 1 byte) and alignment constraint.
> > > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > > >
> > > > The typical use case is a PMD that registers space for an offload
> > > > feature, when the application requests to enable this feature. As
> > > > the space in mbuf is limited, the space should only be reserved if it
> > > > is going to be used (i.e when the application explicitly asks for it).
> > > >
> > > > The registration can be done at any moment, but it is not possible
> > > > to unregister fields or flags for now.
> > > >
> > > > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > > > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > > > ---
> > > >
> > > > v2
> > > >
> > > > * Rebase on top of master: solve conflict with Stephen's patchset
> > > > (packet copy)
> > > > * Add new apis to register a dynamic field/flag at a specific place
> > > > * Add a dump function (sugg by David)
> > > > * Enhance field registration function to select the best offset, keeping
> > > > large aligned zones as much as possible (sugg by Konstantin)
> > > > * Use a size_t and unsigned int instead of int when relevant
> > > > (sugg by Konstantin)
> > > > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > > > (sugg by Konstantin)
> > > > * Remove unused argument in private function (sugg by Konstantin)
> > > > * Fix and simplify locking (sugg by Konstantin)
> > > > * Fix minor typo
> > > >
> > > > rfc -> v1
> > > >
> > > > * Rebase on top of master
> > > > * Change registration API to use a structure instead of
> > > > variables, getting rid of #defines (Stephen's comment)
> > > > * Update flag registration to use a similar API as fields.
> > > > * Change max name length from 32 to 64 (sugg. by Thomas)
> > > > * Enhance API documentation (Haiyue's and Andrew's comments)
> > > > * Add a debug log at registration
> > > > * Add some words in release note
> > > > * Did some performance tests (sugg. by Andrew):
> > > > On my platform, reading a dynamic field takes ~3 cycles more
> > > > than a static field, and ~2 cycles more for writing.
> > > >
> > > > app/test/test_mbuf.c | 145 ++++++-
> > > > doc/guides/rel_notes/release_19_11.rst | 7 +
> > > > lib/librte_mbuf/Makefile | 2 +
> > > > lib/librte_mbuf/meson.build | 6 +-
> > > > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > > > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > > > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > > > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > > > 8 files changed, 959 insertions(+), 5 deletions(-)
> > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> > > >
> > > > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > > > index b9c2b2500..01cafad59 100644
> > > > --- a/app/test/test_mbuf.c
> > > > +++ b/app/test/test_mbuf.c
> > > > @@ -28,6 +28,7 @@
> > > > #include <rte_random.h>
> > >
> > > [snip]
> > >
> > > > +/**
> > > > + * Helper macro to access to a dynamic field.
> > > > + */
> > > > +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) + (offset)))
> > > > +
> > >
> > > The suggested macro is missed ? ;-)
> > > /**
> > > * Helper macro to access to a dynamic flag.
> > > */
> > > #define RTE_MBUF_DYNFLAG(offset) (1ULL << (offset))
> >
> > Yes, sorry.
> >
> > Thinking a bit more about it, I wonder if the macros below aren't
> > more consistent with the dynamic field (because they take the mbuf
> > as parameter)?
> >
> > #define RTE_MBUF_SET_DYNFLAG(m, bitnum, val) ...
> > #define RTE_MBUF_GET_DYNFLAG(m, bitnum) ...
> >
> > They could even be static inline functions.
> >
> > On the other hand, these helpers would be generic to ol_flags, not only
> > for dynamic flags. Today, we use (1ULL << bit) for ol_flags, which makes
> > me wonder... is the macro really needed after all? :)
> >
>
> I used as this:
> 1). in PMD:
> mb->ol_flags |= RTE_MBUF_DYNFLAG(ol_offset);
>
>
> 2). In testpmd
> if (mb->ol_flags & RTE_MBUF_DYNFLAG(ol_offset))
> ...
>
> The above two macros look better in real use.
I just looked at http://patchwork.dpdk.org/patch/60908/
In the patch, a mask is used instead of a bit number, which is indeed
better in terms of performance. This makes the macro not that useful,
given there is a specific helper.
> > > BTW, should we have a place to put the registered dynamic fields and flags
> > > names together (a name overview -- detail Link to --> PMD's help page) ?
> >
> > The centralized place will be in rte_mbuf_dyn.h for fields/flags that can
> > are shared between several dpdk areas. Some libraries/pmd could have private
> > dynamic fields/flags. In any case, I think the same namespace than functions
> > should be used. Probably something like this:
> > - "rte_mbuf_dynfield_<name>" in mbuf lib
> > - "rte_<libname>_dynfield_<name>" in other libs
> > - "rte_net_<pmd>_dynfield_<name>" in pmds
> > - "<name>" in apps
> >
> > > Since rte_mbuf_dynfield:name & rte_mbuf_dynflag:name work as a API style,
> > > users can check how many 'names' registered, developers can check whether
> > > the names they want to use are registered or not ? They don't need to have
> > > to check the rte_errno ... Just a suggestion for user experience.
> >
> > I did not get you point. Does my response above answers to your question?
> >
>
> Yes, the name conversation you mentioned above is a good practice, then no doc
> needed any more, thanks!
>
> > Regards,
> > Olivier
> -----Original Message-----
> From: Olivier Matz [mailto:olivier.matz@6wind.com]
> Sent: Friday, October 18, 2019 17:48
> To: Wang, Haiyue <haiyue.wang@intel.com>
> Cc: dev@dpdk.org; Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: Re: [PATCH v2] mbuf: support dynamic fields and flags
>
> On Fri, Oct 18, 2019 at 08:28:02AM +0000, Wang, Haiyue wrote:
> > Hi Olivier,
> >
> > > -----Original Message-----
> > > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > > Sent: Friday, October 18, 2019 15:54
> > > To: Wang, Haiyue <haiyue.wang@intel.com>
> > > Cc: dev@dpdk.org; Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce
> > > <bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > > <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> > > <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> > > <thomas@monjalon.net>
> > > Subject: Re: [PATCH v2] mbuf: support dynamic fields and flags
> > >
> > > Hi Haiyue,
> > >
> > > On Fri, Oct 18, 2019 at 02:47:50AM +0000, Wang, Haiyue wrote:
> > > > Hi Olivier
> > > >
> > > > > -----Original Message-----
> > > > > From: Olivier Matz [mailto:olivier.matz@6wind.com]
> > > > > Sent: Thursday, October 17, 2019 22:42
> > > > > To: dev@dpdk.org
> > > > > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce
> <bruce.richardson@intel.com>;
> > > Wang,
> > > > > Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > > > > <keith.wiles@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>; Morten Brørup
> > > > > <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> > > > > <thomas@monjalon.net>
> > > > > Subject: [PATCH v2] mbuf: support dynamic fields and flags
> > > > >
> > > > > Many features require to store data inside the mbuf. As the room in mbuf
> > > > > structure is limited, it is not possible to have a field for each
> > > > > feature. Also, changing fields in the mbuf structure can break the API
> > > > > or ABI.
> > > > >
> > > > > This commit addresses these issues, by enabling the dynamic registration
> > > > > of fields or flags:
> > > > >
> > > > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > > > given size (>= 1 byte) and alignment constraint.
> > > > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > > > >
> > > > > The typical use case is a PMD that registers space for an offload
> > > > > feature, when the application requests to enable this feature. As
> > > > > the space in mbuf is limited, the space should only be reserved if it
> > > > > is going to be used (i.e when the application explicitly asks for it).
> > > > >
> > > > > The registration can be done at any moment, but it is not possible
> > > > > to unregister fields or flags for now.
> > > > >
> > > > > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > > > > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > > > > ---
> > > > >
> > > > > v2
> > > > >
> > > > > * Rebase on top of master: solve conflict with Stephen's patchset
> > > > > (packet copy)
> > > > > * Add new apis to register a dynamic field/flag at a specific place
> > > > > * Add a dump function (sugg by David)
> > > > > * Enhance field registration function to select the best offset, keeping
> > > > > large aligned zones as much as possible (sugg by Konstantin)
> > > > > * Use a size_t and unsigned int instead of int when relevant
> > > > > (sugg by Konstantin)
> > > > > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > > > > (sugg by Konstantin)
> > > > > * Remove unused argument in private function (sugg by Konstantin)
> > > > > * Fix and simplify locking (sugg by Konstantin)
> > > > > * Fix minor typo
> > > > >
> > > > > rfc -> v1
> > > > >
> > > > > * Rebase on top of master
> > > > > * Change registration API to use a structure instead of
> > > > > variables, getting rid of #defines (Stephen's comment)
> > > > > * Update flag registration to use a similar API as fields.
> > > > > * Change max name length from 32 to 64 (sugg. by Thomas)
> > > > > * Enhance API documentation (Haiyue's and Andrew's comments)
> > > > > * Add a debug log at registration
> > > > > * Add some words in release note
> > > > > * Did some performance tests (sugg. by Andrew):
> > > > > On my platform, reading a dynamic field takes ~3 cycles more
> > > > > than a static field, and ~2 cycles more for writing.
> > > > >
> > > > > app/test/test_mbuf.c | 145 ++++++-
> > > > > doc/guides/rel_notes/release_19_11.rst | 7 +
> > > > > lib/librte_mbuf/Makefile | 2 +
> > > > > lib/librte_mbuf/meson.build | 6 +-
> > > > > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > > > > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > > > > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > > > > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > > > > 8 files changed, 959 insertions(+), 5 deletions(-)
> > > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> > > > >
> > > > > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > > > > index b9c2b2500..01cafad59 100644
> > > > > --- a/app/test/test_mbuf.c
> > > > > +++ b/app/test/test_mbuf.c
> > > > > @@ -28,6 +28,7 @@
> > > > > #include <rte_random.h>
> > > >
> > > > [snip]
> > > >
> > > > > +/**
> > > > > + * Helper macro to access to a dynamic field.
> > > > > + */
> > > > > +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) + (offset)))
> > > > > +
> > > >
> > > > The suggested macro is missed ? ;-)
> > > > /**
> > > > * Helper macro to access to a dynamic flag.
> > > > */
> > > > #define RTE_MBUF_DYNFLAG(offset) (1ULL << (offset))
> > >
> > > Yes, sorry.
> > >
> > > Thinking a bit more about it, I wonder if the macros below aren't
> > > more consistent with the dynamic field (because they take the mbuf
> > > as parameter)?
> > >
> > > #define RTE_MBUF_SET_DYNFLAG(m, bitnum, val) ...
> > > #define RTE_MBUF_GET_DYNFLAG(m, bitnum) ...
> > >
> > > They could even be static inline functions.
> > >
> > > On the other hand, these helpers would be generic to ol_flags, not only
> > > for dynamic flags. Today, we use (1ULL << bit) for ol_flags, which makes
> > > me wonder... is the macro really needed after all? :)
> > >
> >
> > I used as this:
> > 1). in PMD:
> > mb->ol_flags |= RTE_MBUF_DYNFLAG(ol_offset);
> >
> >
> > 2). In testpmd
> > if (mb->ol_flags & RTE_MBUF_DYNFLAG(ol_offset))
> > ...
> >
> > The above two macros look better in real use.
>
> I just looked at http://patchwork.dpdk.org/patch/60908/
> In the patch, a mask is used instead of a bit number, which is indeed
> better in terms of performance. This makes the macro not that useful,
> given there is a specific helper.
>
'a mask is used instead of a bit number' good practice, yes, then no need
this macro, thanks for sharing. ;-)
> Many features require to store data inside the mbuf. As the room in mbuf
> structure is limited, it is not possible to have a field for each
> feature. Also, changing fields in the mbuf structure can break the API
> or ABI.
>
> This commit addresses these issues, by enabling the dynamic registration
> of fields or flags:
>
> - a dynamic field is a named area in the rte_mbuf structure, with a
> given size (>= 1 byte) and alignment constraint.
> - a dynamic flag is a named bit in the rte_mbuf structure.
>
> The typical use case is a PMD that registers space for an offload
> feature, when the application requests to enable this feature. As
> the space in mbuf is limited, the space should only be reserved if it
> is going to be used (i.e when the application explicitly asks for it).
>
> The registration can be done at any moment, but it is not possible
> to unregister fields or flags for now.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>
> v2
>
> * Rebase on top of master: solve conflict with Stephen's patchset
> (packet copy)
> * Add new apis to register a dynamic field/flag at a specific place
> * Add a dump function (sugg by David)
> * Enhance field registration function to select the best offset, keeping
> large aligned zones as much as possible (sugg by Konstantin)
> * Use a size_t and unsigned int instead of int when relevant
> (sugg by Konstantin)
> * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> (sugg by Konstantin)
> * Remove unused argument in private function (sugg by Konstantin)
> * Fix and simplify locking (sugg by Konstantin)
> * Fix minor typo
>
> rfc -> v1
>
> * Rebase on top of master
> * Change registration API to use a structure instead of
> variables, getting rid of #defines (Stephen's comment)
> * Update flag registration to use a similar API as fields.
> * Change max name length from 32 to 64 (sugg. by Thomas)
> * Enhance API documentation (Haiyue's and Andrew's comments)
> * Add a debug log at registration
> * Add some words in release note
> * Did some performance tests (sugg. by Andrew):
> On my platform, reading a dynamic field takes ~3 cycles more
> than a static field, and ~2 cycles more for writing.
>
> app/test/test_mbuf.c | 145 ++++++-
> doc/guides/rel_notes/release_19_11.rst | 7 +
> lib/librte_mbuf/Makefile | 2 +
> lib/librte_mbuf/meson.build | 6 +-
> lib/librte_mbuf/rte_mbuf.h | 23 +-
> lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> lib/librte_mbuf/rte_mbuf_version.map | 7 +
> 8 files changed, 959 insertions(+), 5 deletions(-)
> create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
>
> diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> index b9c2b2500..01cafad59 100644
> --- a/app/test/test_mbuf.c
> +++ b/app/test/test_mbuf.c
> @@ -28,6 +28,7 @@
> #include <rte_random.h>
> #include <rte_cycles.h>
> #include <rte_malloc.h>
> +#include <rte_mbuf_dyn.h>
>
> #include "test.h"
>
> @@ -657,7 +658,6 @@ test_attach_from_different_pool(struct rte_mempool *pktmbuf_pool,
> rte_pktmbuf_free(clone2);
> return -1;
> }
> -#undef GOTO_FAIL
>
> /*
> * test allocation and free of mbufs
> @@ -1276,6 +1276,143 @@ test_tx_offload(void)
> return (v1 == v2) ? 0 : -EINVAL;
> }
>
> +static int
> +test_mbuf_dyn(struct rte_mempool *pktmbuf_pool)
> +{
> + const struct rte_mbuf_dynfield dynfield = {
> + .name = "test-dynfield",
> + .size = sizeof(uint8_t),
> + .align = __alignof__(uint8_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield2 = {
> + .name = "test-dynfield2",
> + .size = sizeof(uint16_t),
> + .align = __alignof__(uint16_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield3 = {
> + .name = "test-dynfield3",
> + .size = sizeof(uint8_t),
> + .align = __alignof__(uint8_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield_fail_big = {
> + .name = "test-dynfield-fail-big",
> + .size = 256,
> + .align = 1,
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield_fail_align = {
> + .name = "test-dynfield-fail-align",
> + .size = 1,
> + .align = 3,
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag = {
> + .name = "test-dynflag",
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag2 = {
> + .name = "test-dynflag2",
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag3 = {
> + .name = "test-dynflag3",
> + .flags = 0,
> + };
> + struct rte_mbuf *m = NULL;
> + int offset, offset2, offset3;
> + int flag, flag2, flag3;
> + int ret;
> +
> + printf("Test mbuf dynamic fields and flags\n");
> + rte_mbuf_dyn_dump(stdout);
> +
> + offset = rte_mbuf_dynfield_register(&dynfield);
> + if (offset == -1)
> + GOTO_FAIL("failed to register dynamic field, offset=%d: %s",
> + offset, strerror(errno));
> +
> + ret = rte_mbuf_dynfield_register(&dynfield);
> + if (ret != offset)
> + GOTO_FAIL("failed to lookup dynamic field, ret=%d: %s",
> + ret, strerror(errno));
> +
> + offset2 = rte_mbuf_dynfield_register(&dynfield2);
> + if (offset2 == -1 || offset2 == offset || (offset2 & 1))
> + GOTO_FAIL("failed to register dynamic field 2, offset2=%d: %s",
> + offset2, strerror(errno));
> +
> + offset3 = rte_mbuf_dynfield_register_offset(&dynfield3,
> + offsetof(struct rte_mbuf, dynfield1[1]));
> + if (offset3 != offsetof(struct rte_mbuf, dynfield1[1]))
> + GOTO_FAIL("failed to register dynamic field 3, offset=%d: %s",
> + offset3, strerror(errno));
> +
> + printf("dynfield: offset=%d, offset2=%d, offset3=%d\n",
> + offset, offset2, offset3);
> +
> + ret = rte_mbuf_dynfield_register(&dynfield_fail_big);
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (too big)");
> +
> + ret = rte_mbuf_dynfield_register(&dynfield_fail_align);
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (bad alignment)");
> +
> + ret = rte_mbuf_dynfield_register_offset(&dynfield_fail_align,
> + offsetof(struct rte_mbuf, ol_flags));
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (not avail)");
> +
> + flag = rte_mbuf_dynflag_register(&dynflag);
> + if (flag == -1)
> + GOTO_FAIL("failed to register dynamic flag, flag=%d: %s",
> + flag, strerror(errno));
> +
> + ret = rte_mbuf_dynflag_register(&dynflag);
> + if (ret != flag)
> + GOTO_FAIL("failed to lookup dynamic flag, ret=%d: %s",
> + ret, strerror(errno));
> +
> + flag2 = rte_mbuf_dynflag_register(&dynflag2);
> + if (flag2 == -1 || flag2 == flag)
> + GOTO_FAIL("failed to register dynamic flag 2, flag2=%d: %s",
> + flag2, strerror(errno));
> +
> + flag3 = rte_mbuf_dynflag_register_bitnum(&dynflag3,
> + rte_bsf64(PKT_LAST_FREE));
> + if (flag3 != rte_bsf64(PKT_LAST_FREE))
> + GOTO_FAIL("failed to register dynamic flag 3, flag2=%d: %s",
> + flag3, strerror(errno));
> +
> + printf("dynflag: flag=%d, flag2=%d, flag3=%d\n", flag, flag2, flag3);
> +
> + /* set, get dynamic field */
> + m = rte_pktmbuf_alloc(pktmbuf_pool);
> + if (m == NULL)
> + GOTO_FAIL("Cannot allocate mbuf");
> +
> + *RTE_MBUF_DYNFIELD(m, offset, uint8_t *) = 1;
> + if (*RTE_MBUF_DYNFIELD(m, offset, uint8_t *) != 1)
> + GOTO_FAIL("failed to read dynamic field");
> + *RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) = 1000;
> + if (*RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) != 1000)
> + GOTO_FAIL("failed to read dynamic field");
> +
> + /* set a dynamic flag */
> + m->ol_flags |= (1ULL << flag);
> +
> + rte_mbuf_dyn_dump(stdout);
> + rte_pktmbuf_free(m);
> + return 0;
> +fail:
> + rte_pktmbuf_free(m);
> + return -1;
> +}
> +#undef GOTO_FAIL
> +
> static int
> test_mbuf(void)
> {
> @@ -1295,6 +1432,12 @@ test_mbuf(void)
> goto err;
> }
>
> + /* test registration of dynamic fields and flags */
> + if (test_mbuf_dyn(pktmbuf_pool) < 0) {
> + printf("mbuf dynflag test failed\n");
> + goto err;
> + }
> +
> /* create a specific pktmbuf pool with a priv_size != 0 and no data
> * room size */
> pktmbuf_pool2 = rte_pktmbuf_pool_create("test_pktmbuf_pool2",
> diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
> index 85953b962..9e9c94554 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -21,6 +21,13 @@ DPDK Release 19.11
>
> xdg-open build/doc/html/guides/rel_notes/release_19_11.html
>
> +* **Add support of support dynamic fields and flags in mbuf.**
> +
> + This new feature adds the ability to dynamically register some room
> + for a field or a flag in the mbuf structure. This is typically used
> + for specific offload features, where adding a static field or flag
> + in the mbuf is not justified.
> +
>
> New Features
> ------------
> diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile
> index c8f6d2689..5a9bcee73 100644
> --- a/lib/librte_mbuf/Makefile
> +++ b/lib/librte_mbuf/Makefile
> @@ -17,8 +17,10 @@ LIBABIVER := 5
>
> # all source are stored in SRCS-y
> SRCS-$(CONFIG_RTE_LIBRTE_MBUF) := rte_mbuf.c rte_mbuf_ptype.c rte_mbuf_pool_ops.c
> +SRCS-$(CONFIG_RTE_LIBRTE_MBUF) += rte_mbuf_dyn.c
>
> # install includes
> SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include := rte_mbuf.h rte_mbuf_ptype.h rte_mbuf_pool_ops.h
> +SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include += rte_mbuf_dyn.h
>
> include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/lib/librte_mbuf/meson.build b/lib/librte_mbuf/meson.build
> index 6cc11ebb4..9137e8f26 100644
> --- a/lib/librte_mbuf/meson.build
> +++ b/lib/librte_mbuf/meson.build
> @@ -2,8 +2,10 @@
> # Copyright(c) 2017 Intel Corporation
>
> version = 5
> -sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c')
> -headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h')
> +sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c',
> + 'rte_mbuf_dyn.c')
> +headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h',
> + 'rte_mbuf_dyn.h')
> deps += ['mempool']
>
> allow_experimental_apis = true
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> index fb0849ac1..5740b1e93 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -198,9 +198,12 @@ extern "C" {
> #define PKT_RX_OUTER_L4_CKSUM_GOOD (1ULL << 22)
> #define PKT_RX_OUTER_L4_CKSUM_INVALID ((1ULL << 21) | (1ULL << 22))
>
> -/* add new RX flags here */
> +/* add new RX flags here, don't forget to update PKT_FIRST_FREE */
>
> -/* add new TX flags here */
> +#define PKT_FIRST_FREE (1ULL << 23)
> +#define PKT_LAST_FREE (1ULL << 39)
> +
> +/* add new TX flags here, don't forget to update PKT_LAST_FREE */
>
> /**
> * Indicate that the metadata field in the mbuf is in use.
> @@ -738,6 +741,7 @@ struct rte_mbuf {
> */
> struct rte_mbuf_ext_shared_info *shinfo;
>
> + uint64_t dynfield1[2]; /**< Reserved for dynamic fields. */
> } __rte_cache_aligned;
>
> /**
> @@ -1684,6 +1688,20 @@ rte_pktmbuf_attach_extbuf(struct rte_mbuf *m, void *buf_addr,
> */
> #define rte_pktmbuf_detach_extbuf(m) rte_pktmbuf_detach(m)
>
> +/**
> + * Copy dynamic fields from m_src to m_dst.
> + *
> + * @param m_dst
> + * The destination mbuf.
> + * @param m_src
> + * The source mbuf.
> + */
> +static inline void
> +rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> +{
> + memcpy(&mdst->dynfield1, msrc->dynfield1, sizeof(mdst->dynfield1));
> +}
> +
> /* internal */
> static inline void
> __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> @@ -1695,6 +1713,7 @@ __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> mdst->hash = msrc->hash;
> mdst->packet_type = msrc->packet_type;
> mdst->timestamp = msrc->timestamp;
> + rte_mbuf_dynfield_copy(mdst, msrc);
> }
>
> /**
> diff --git a/lib/librte_mbuf/rte_mbuf_dyn.c b/lib/librte_mbuf/rte_mbuf_dyn.c
> new file mode 100644
> index 000000000..9ef235483
> --- /dev/null
> +++ b/lib/librte_mbuf/rte_mbuf_dyn.c
> @@ -0,0 +1,548 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 6WIND S.A.
> + */
> +
> +#include <sys/queue.h>
> +#include <stdint.h>
> +#include <limits.h>
> +
> +#include <rte_common.h>
> +#include <rte_eal.h>
> +#include <rte_eal_memconfig.h>
> +#include <rte_tailq.h>
> +#include <rte_errno.h>
> +#include <rte_malloc.h>
> +#include <rte_string_fns.h>
> +#include <rte_mbuf.h>
> +#include <rte_mbuf_dyn.h>
> +
> +#define RTE_MBUF_DYN_MZNAME "rte_mbuf_dyn"
> +
> +struct mbuf_dynfield_elt {
> + TAILQ_ENTRY(mbuf_dynfield_elt) next;
> + struct rte_mbuf_dynfield params;
> + size_t offset;
> +};
> +TAILQ_HEAD(mbuf_dynfield_list, rte_tailq_entry);
> +
> +static struct rte_tailq_elem mbuf_dynfield_tailq = {
> + .name = "RTE_MBUF_DYNFIELD",
> +};
> +EAL_REGISTER_TAILQ(mbuf_dynfield_tailq);
> +
> +struct mbuf_dynflag_elt {
> + TAILQ_ENTRY(mbuf_dynflag_elt) next;
> + struct rte_mbuf_dynflag params;
> + unsigned int bitnum;
> +};
> +TAILQ_HEAD(mbuf_dynflag_list, rte_tailq_entry);
> +
> +static struct rte_tailq_elem mbuf_dynflag_tailq = {
> + .name = "RTE_MBUF_DYNFLAG",
> +};
> +EAL_REGISTER_TAILQ(mbuf_dynflag_tailq);
> +
> +struct mbuf_dyn_shm {
> + /**
> + * For each mbuf byte, free_space[i] != 0 if space is free.
> + * The value is the size of the biggest aligned element that
> + * can fit in the zone.
> + */
> + uint8_t free_space[sizeof(struct rte_mbuf)];
> + /** Bitfield of available flags. */
> + uint64_t free_flags;
> +};
> +static struct mbuf_dyn_shm *shm;
> +
> +/* Set the value of free_space[] according to the size and alignment of
> + * the free areas. This helps to select the best place when reserving a
> + * dynamic field. Assume tailq is locked.
> + */
> +static void
> +process_score(void)
> +{
> + size_t off, align, size, i;
> +
> + /* first, erase previous info */
> + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> + if (shm->free_space[i])
> + shm->free_space[i] = 1;
> + }
> +
> + for (off = 0; off < sizeof(struct rte_mbuf); off++) {
> + /* get the size of the free zone */
> + for (size = 0; shm->free_space[off + size]; size++)
> + ;
> + if (size == 0)
> + continue;
> +
> + /* get the alignment of biggest object that can fit in
> + * the zone at this offset.
> + */
> + for (align = 1;
> + (off % (align << 1)) == 0 && (align << 1) <= size;
> + align <<= 1)
> + ;
> +
> + /* save it in free_space[] */
> + for (i = off; i < off + size; i++)
> + shm->free_space[i] = RTE_MAX(align, shm->free_space[i]);
> + }
> +}
> +
> +/* Allocate and initialize the shared memory. Assume tailq is locked */
> +static int
> +init_shared_mem(void)
> +{
> + const struct rte_memzone *mz;
> + uint64_t mask;
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + mz = rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
> + sizeof(struct mbuf_dyn_shm),
> + SOCKET_ID_ANY, 0,
> + RTE_CACHE_LINE_SIZE);
> + } else {
> + mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
> + }
> + if (mz == NULL)
> + return -1;
> +
> + shm = mz->addr;
> +
> +#define mark_free(field) \
> + memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
> + 1, sizeof(((struct rte_mbuf *)0)->field))
Still think it would look nicer without multi-line macro defines/undef in the middle of the function.
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + /* init free_space, keep it sync'd with
> + * rte_mbuf_dynfield_copy().
> + */
> + memset(shm, 0, sizeof(*shm));
> + mark_free(dynfield1);
> +
> + /* init free_flags */
> + for (mask = PKT_FIRST_FREE; mask <= PKT_LAST_FREE; mask <<= 1)
> + shm->free_flags |= mask;
> +
> + process_score();
> + }
> +#undef mark_free
> +
> + return 0;
> +}
> +
> +/* check if this offset can be used */
> +static int
> +check_offset(size_t offset, size_t size, size_t align)
> +{
> + size_t i;
> +
> + if ((offset & (align - 1)) != 0)
> + return -1;
> + if (offset + size > sizeof(struct rte_mbuf))
> + return -1;
> +
> + for (i = 0; i < size; i++) {
> + if (!shm->free_space[i + offset])
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static struct mbuf_dynfield_elt *
> +__mbuf_dynfield_lookup(const char *name)
> +{
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *mbuf_dynfield;
> + struct rte_tailq_entry *te;
> +
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> +
> + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> + mbuf_dynfield = (struct mbuf_dynfield_elt *)te->data;
> + if (strcmp(name, mbuf_dynfield->params.name) == 0)
> + break;
> + }
> +
> + if (te == NULL) {
> + rte_errno = ENOENT;
> + return NULL;
> + }
> +
> + return mbuf_dynfield;
> +}
> +
> +int
> +rte_mbuf_dynfield_lookup(const char *name, struct rte_mbuf_dynfield *params)
> +{
> + struct mbuf_dynfield_elt *mbuf_dynfield;
> +
> + if (shm == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_read_lock();
> + mbuf_dynfield = __mbuf_dynfield_lookup(name);
> + rte_mcfg_tailq_read_unlock();
> +
> + if (mbuf_dynfield == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + if (params != NULL)
> + memcpy(params, &mbuf_dynfield->params, sizeof(*params));
> +
> + return mbuf_dynfield->offset;
> +}
> +
> +static int mbuf_dynfield_cmp(const struct rte_mbuf_dynfield *params1,
> + const struct rte_mbuf_dynfield *params2)
> +{
> + if (strcmp(params1->name, params2->name))
> + return -1;
> + if (params1->size != params2->size)
> + return -1;
> + if (params1->align != params2->align)
> + return -1;
> + if (params1->flags != params2->flags)
> + return -1;
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static int
> +__rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params,
> + size_t req)
> +{
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *mbuf_dynfield = NULL;
> + struct rte_tailq_entry *te = NULL;
> + unsigned int best_zone = UINT_MAX;
> + size_t i, offset;
> + int ret;
> +
> + if (shm == NULL && init_shared_mem() < 0)
> + return -1;
> +
> + mbuf_dynfield = __mbuf_dynfield_lookup(params->name);
> + if (mbuf_dynfield != NULL) {
> + if (req != SIZE_MAX && req != mbuf_dynfield->offset) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + if (mbuf_dynfield_cmp(params, &mbuf_dynfield->params) < 0) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + return mbuf_dynfield->offset;
> + }
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> + rte_errno = EPERM;
> + return -1;
> + }
> +
> + if (req == SIZE_MAX) {
> + for (offset = 0;
> + offset < sizeof(struct rte_mbuf);
> + offset++) {
> + if (check_offset(offset, params->size,
> + params->align) == 0 &&
> + shm->free_space[offset] < best_zone) {
Probably worth to explain a bit more here about best_zone logic -
trying to find offset with minimal score (minimal continuous length), etc.
> + best_zone = shm->free_space[offset];
> + req = offset;
> + }
> + }
> + if (req == SIZE_MAX) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> + } else {
> + if (check_offset(req, params->size, params->align) < 0) {
> + rte_errno = EBUSY;
> + return -1;
> + }
> + }
> +
> + offset = req;
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> +
> + te = rte_zmalloc("MBUF_DYNFIELD_TAILQ_ENTRY", sizeof(*te), 0);
> + if (te == NULL)
> + return -1;
> +
> + mbuf_dynfield = rte_zmalloc("mbuf_dynfield", sizeof(*mbuf_dynfield), 0);
> + if (mbuf_dynfield == NULL) {
> + rte_free(te);
> + return -1;
> + }
> +
> + ret = strlcpy(mbuf_dynfield->params.name, params->name,
> + sizeof(mbuf_dynfield->params.name));
> + if (ret < 0 || ret >= (int)sizeof(mbuf_dynfield->params.name)) {
> + rte_errno = ENAMETOOLONG;
> + rte_free(mbuf_dynfield);
> + rte_free(te);
> + return -1;
> + }
> + memcpy(&mbuf_dynfield->params, params, sizeof(mbuf_dynfield->params));
> + mbuf_dynfield->offset = offset;
> + te->data = mbuf_dynfield;
> +
> + TAILQ_INSERT_TAIL(mbuf_dynfield_list, te, next);
> +
> + for (i = offset; i < offset + params->size; i++)
> + shm->free_space[i] = 0;
> + process_score();
> +
> + RTE_LOG(DEBUG, MBUF, "Registered dynamic field %s (sz=%zu, al=%zu, fl=0x%x) -> %zd\n",
> + params->name, params->size, params->align, params->flags,
> + offset);
> +
> + return offset;
> +}
> +
> +int
> +rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params,
> + size_t req)
> +{
> + int ret;
> +
> + if (params->size >= sizeof(struct rte_mbuf)) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (!rte_is_power_of_2(params->align)) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (params->flags != 0) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_write_lock();
> + ret = __rte_mbuf_dynfield_register_offset(params, req);
> + rte_mcfg_tailq_write_unlock();
> +
> + return ret;
> +}
> +
> +int
> +rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params)
> +{
> + return rte_mbuf_dynfield_register_offset(params, SIZE_MAX);
> +}
> +
> +/* assume tailq is locked */
> +static struct mbuf_dynflag_elt *
> +__mbuf_dynflag_lookup(const char *name)
> +{
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *mbuf_dynflag;
> + struct rte_tailq_entry *te;
> +
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> +
> + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> + mbuf_dynflag = (struct mbuf_dynflag_elt *)te->data;
> + if (strncmp(name, mbuf_dynflag->params.name,
> + RTE_MBUF_DYN_NAMESIZE) == 0)
> + break;
> + }
> +
> + if (te == NULL) {
> + rte_errno = ENOENT;
> + return NULL;
> + }
> +
> + return mbuf_dynflag;
> +}
> +
> +int
> +rte_mbuf_dynflag_lookup(const char *name,
> + struct rte_mbuf_dynflag *params)
> +{
> + struct mbuf_dynflag_elt *mbuf_dynflag;
> +
> + if (shm == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_read_lock();
> + mbuf_dynflag = __mbuf_dynflag_lookup(name);
> + rte_mcfg_tailq_read_unlock();
> +
> + if (mbuf_dynflag == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + if (params != NULL)
> + memcpy(params, &mbuf_dynflag->params, sizeof(*params));
> +
> + return mbuf_dynflag->bitnum;
> +}
> +
> +static int mbuf_dynflag_cmp(const struct rte_mbuf_dynflag *params1,
> + const struct rte_mbuf_dynflag *params2)
> +{
> + if (strcmp(params1->name, params2->name))
> + return -1;
> + if (params1->flags != params2->flags)
> + return -1;
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static int
> +__rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> + unsigned int req)
> +{
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *mbuf_dynflag = NULL;
> + struct rte_tailq_entry *te = NULL;
> + unsigned int bitnum;
> + int ret;
> +
> + if (shm == NULL && init_shared_mem() < 0)
> + return -1;
> +
> + mbuf_dynflag = __mbuf_dynflag_lookup(params->name);
> + if (mbuf_dynflag != NULL) {
> + if (req != UINT_MAX && req != mbuf_dynflag->bitnum) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + if (mbuf_dynflag_cmp(params, &mbuf_dynflag->params) < 0) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + return mbuf_dynflag->bitnum;
> + }
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> + rte_errno = EPERM;
> + return -1;
> + }
> +
> + if (req == UINT_MAX) {
> + if (shm->free_flags == 0) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> + bitnum = rte_bsf64(shm->free_flags);
> + } else {
> + if ((shm->free_flags & (1ULL << req)) == 0) {
> + rte_errno = EBUSY;
> + return -1;
> + }
> + bitnum = req;
> + }
> +
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> +
> + te = rte_zmalloc("MBUF_DYNFLAG_TAILQ_ENTRY", sizeof(*te), 0);
> + if (te == NULL)
> + return -1;
> +
> + mbuf_dynflag = rte_zmalloc("mbuf_dynflag", sizeof(*mbuf_dynflag), 0);
> + if (mbuf_dynflag == NULL) {
> + rte_free(te);
> + return -1;
> + }
> +
> + ret = strlcpy(mbuf_dynflag->params.name, params->name,
> + sizeof(mbuf_dynflag->params.name));
> + if (ret < 0 || ret >= (int)sizeof(mbuf_dynflag->params.name)) {
> + rte_free(mbuf_dynflag);
> + rte_free(te);
> + rte_errno = ENAMETOOLONG;
> + return -1;
> + }
> + mbuf_dynflag->bitnum = bitnum;
> + te->data = mbuf_dynflag;
> +
> + TAILQ_INSERT_TAIL(mbuf_dynflag_list, te, next);
> +
> + shm->free_flags &= ~(1ULL << bitnum);
> +
> + RTE_LOG(DEBUG, MBUF, "Registered dynamic flag %s (fl=0x%x) -> %u\n",
> + params->name, params->flags, bitnum);
> +
> + return bitnum;
> +}
> +
> +int
> +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> + unsigned int req)
> +{
> + int ret;
> +
> + if (req != UINT_MAX && req >= 64) {
Might be better to replace 64 with something like sizeof(mbuf->ol_flags) * CHAR_BIT or so.
Apart from that:
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> + rte_errno = EINVAL;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_write_lock();
> + ret = __rte_mbuf_dynflag_register_bitnum(params, req);
> + rte_mcfg_tailq_write_unlock();
> +
> + return ret;
> +}
> +
> -----Original Message-----
> From: Ananyev, Konstantin
> Sent: Wednesday, October 23, 2019 06:52
> To: Olivier Matz <olivier.matz@6wind.com>; dev@dpdk.org
> Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>; Wang,
> Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> <keith.wiles@intel.com>; Morten Brørup <mb@smartsharesystems.com>; Stephen Hemminger
> <stephen@networkplumber.org>; Thomas Monjalon <thomas@monjalon.net>
> Subject: RE: [PATCH v2] mbuf: support dynamic fields and flags
>
>
> > Many features require to store data inside the mbuf. As the room in mbuf
> > structure is limited, it is not possible to have a field for each
> > feature. Also, changing fields in the mbuf structure can break the API
> > or ABI.
> >
> > This commit addresses these issues, by enabling the dynamic registration
> > of fields or flags:
> >
> > - a dynamic field is a named area in the rte_mbuf structure, with a
> > given size (>= 1 byte) and alignment constraint.
> > - a dynamic flag is a named bit in the rte_mbuf structure.
> >
> > The typical use case is a PMD that registers space for an offload
> > feature, when the application requests to enable this feature. As
> > the space in mbuf is limited, the space should only be reserved if it
> > is going to be used (i.e when the application explicitly asks for it).
> >
> > The registration can be done at any moment, but it is not possible
> > to unregister fields or flags for now.
> >
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> >
> > v2
> >
> > * Rebase on top of master: solve conflict with Stephen's patchset
> > (packet copy)
> > * Add new apis to register a dynamic field/flag at a specific place
> > * Add a dump function (sugg by David)
> > * Enhance field registration function to select the best offset, keeping
> > large aligned zones as much as possible (sugg by Konstantin)
> > * Use a size_t and unsigned int instead of int when relevant
> > (sugg by Konstantin)
> > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > (sugg by Konstantin)
> > * Remove unused argument in private function (sugg by Konstantin)
> > * Fix and simplify locking (sugg by Konstantin)
> > * Fix minor typo
> >
> > rfc -> v1
> >
> > * Rebase on top of master
> > * Change registration API to use a structure instead of
> > variables, getting rid of #defines (Stephen's comment)
> > * Update flag registration to use a similar API as fields.
> > * Change max name length from 32 to 64 (sugg. by Thomas)
> > * Enhance API documentation (Haiyue's and Andrew's comments)
> > * Add a debug log at registration
> > * Add some words in release note
> > * Did some performance tests (sugg. by Andrew):
> > On my platform, reading a dynamic field takes ~3 cycles more
> > than a static field, and ~2 cycles more for writing.
> >
> > app/test/test_mbuf.c | 145 ++++++-
> > doc/guides/rel_notes/release_19_11.rst | 7 +
> > lib/librte_mbuf/Makefile | 2 +
> > lib/librte_mbuf/meson.build | 6 +-
> > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > 8 files changed, 959 insertions(+), 5 deletions(-)
> > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> >
> > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > index b9c2b2500..01cafad59 100644
> > --- a/app/test/test_mbuf.c
> > +++ b/app/test/test_mbuf.c
> > @@ -28,6 +28,7 @@
> > #include <rte_random.h>
> > #include <rte_cycles.h>
> > #include <rte_malloc.h>
> > +#include <rte_mbuf_dyn.h>
> >
[snip]
> > +int
> > +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > + unsigned int req)
> > +{
> > + int ret;
> > +
> > + if (req != UINT_MAX && req >= 64) {
>
> Might be better to replace 64 with something like sizeof(mbuf->ol_flags) * CHAR_BIT or so.
Might introduce a new macro like kernel:
/**
* FIELD_SIZEOF - get the size of a struct's field
* @t: the target struct
* @f: the target struct's field
* Return: the size of @f in the struct definition without having a
* declared instance of @t.
*/
#define FIELD_SIZEOF(t, f) (sizeof(((t*)0)->f))
Then: FIELD_SIZEOF(rte_mbuf, ol_flags) * CHAR_BIT
> Apart from that:
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
>
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_write_lock();
> > + ret = __rte_mbuf_dynflag_register_bitnum(params, req);
> > + rte_mcfg_tailq_write_unlock();
> > +
> > + return ret;
> > +}
> > +
On Tue, Oct 22, 2019 at 10:51:51PM +0000, Ananyev, Konstantin wrote:
>
> > Many features require to store data inside the mbuf. As the room in mbuf
> > structure is limited, it is not possible to have a field for each
> > feature. Also, changing fields in the mbuf structure can break the API
> > or ABI.
> >
> > This commit addresses these issues, by enabling the dynamic registration
> > of fields or flags:
> >
> > - a dynamic field is a named area in the rte_mbuf structure, with a
> > given size (>= 1 byte) and alignment constraint.
> > - a dynamic flag is a named bit in the rte_mbuf structure.
> >
> > The typical use case is a PMD that registers space for an offload
> > feature, when the application requests to enable this feature. As
> > the space in mbuf is limited, the space should only be reserved if it
> > is going to be used (i.e when the application explicitly asks for it).
> >
> > The registration can be done at any moment, but it is not possible
> > to unregister fields or flags for now.
> >
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> >
> > v2
> >
> > * Rebase on top of master: solve conflict with Stephen's patchset
> > (packet copy)
> > * Add new apis to register a dynamic field/flag at a specific place
> > * Add a dump function (sugg by David)
> > * Enhance field registration function to select the best offset, keeping
> > large aligned zones as much as possible (sugg by Konstantin)
> > * Use a size_t and unsigned int instead of int when relevant
> > (sugg by Konstantin)
> > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > (sugg by Konstantin)
> > * Remove unused argument in private function (sugg by Konstantin)
> > * Fix and simplify locking (sugg by Konstantin)
> > * Fix minor typo
> >
> > rfc -> v1
> >
> > * Rebase on top of master
> > * Change registration API to use a structure instead of
> > variables, getting rid of #defines (Stephen's comment)
> > * Update flag registration to use a similar API as fields.
> > * Change max name length from 32 to 64 (sugg. by Thomas)
> > * Enhance API documentation (Haiyue's and Andrew's comments)
> > * Add a debug log at registration
> > * Add some words in release note
> > * Did some performance tests (sugg. by Andrew):
> > On my platform, reading a dynamic field takes ~3 cycles more
> > than a static field, and ~2 cycles more for writing.
> >
> > app/test/test_mbuf.c | 145 ++++++-
> > doc/guides/rel_notes/release_19_11.rst | 7 +
> > lib/librte_mbuf/Makefile | 2 +
> > lib/librte_mbuf/meson.build | 6 +-
> > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > 8 files changed, 959 insertions(+), 5 deletions(-)
> > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> >
> > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > index b9c2b2500..01cafad59 100644
> > --- a/app/test/test_mbuf.c
> > +++ b/app/test/test_mbuf.c
> > @@ -28,6 +28,7 @@
> > #include <rte_random.h>
> > #include <rte_cycles.h>
> > #include <rte_malloc.h>
> > +#include <rte_mbuf_dyn.h>
> >
> > #include "test.h"
> >
> > @@ -657,7 +658,6 @@ test_attach_from_different_pool(struct rte_mempool *pktmbuf_pool,
> > rte_pktmbuf_free(clone2);
> > return -1;
> > }
> > -#undef GOTO_FAIL
> >
> > /*
> > * test allocation and free of mbufs
> > @@ -1276,6 +1276,143 @@ test_tx_offload(void)
> > return (v1 == v2) ? 0 : -EINVAL;
> > }
> >
> > +static int
> > +test_mbuf_dyn(struct rte_mempool *pktmbuf_pool)
> > +{
> > + const struct rte_mbuf_dynfield dynfield = {
> > + .name = "test-dynfield",
> > + .size = sizeof(uint8_t),
> > + .align = __alignof__(uint8_t),
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield2 = {
> > + .name = "test-dynfield2",
> > + .size = sizeof(uint16_t),
> > + .align = __alignof__(uint16_t),
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield3 = {
> > + .name = "test-dynfield3",
> > + .size = sizeof(uint8_t),
> > + .align = __alignof__(uint8_t),
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield_fail_big = {
> > + .name = "test-dynfield-fail-big",
> > + .size = 256,
> > + .align = 1,
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield_fail_align = {
> > + .name = "test-dynfield-fail-align",
> > + .size = 1,
> > + .align = 3,
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynflag dynflag = {
> > + .name = "test-dynflag",
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynflag dynflag2 = {
> > + .name = "test-dynflag2",
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynflag dynflag3 = {
> > + .name = "test-dynflag3",
> > + .flags = 0,
> > + };
> > + struct rte_mbuf *m = NULL;
> > + int offset, offset2, offset3;
> > + int flag, flag2, flag3;
> > + int ret;
> > +
> > + printf("Test mbuf dynamic fields and flags\n");
> > + rte_mbuf_dyn_dump(stdout);
> > +
> > + offset = rte_mbuf_dynfield_register(&dynfield);
> > + if (offset == -1)
> > + GOTO_FAIL("failed to register dynamic field, offset=%d: %s",
> > + offset, strerror(errno));
> > +
> > + ret = rte_mbuf_dynfield_register(&dynfield);
> > + if (ret != offset)
> > + GOTO_FAIL("failed to lookup dynamic field, ret=%d: %s",
> > + ret, strerror(errno));
> > +
> > + offset2 = rte_mbuf_dynfield_register(&dynfield2);
> > + if (offset2 == -1 || offset2 == offset || (offset2 & 1))
> > + GOTO_FAIL("failed to register dynamic field 2, offset2=%d: %s",
> > + offset2, strerror(errno));
> > +
> > + offset3 = rte_mbuf_dynfield_register_offset(&dynfield3,
> > + offsetof(struct rte_mbuf, dynfield1[1]));
> > + if (offset3 != offsetof(struct rte_mbuf, dynfield1[1]))
> > + GOTO_FAIL("failed to register dynamic field 3, offset=%d: %s",
> > + offset3, strerror(errno));
> > +
> > + printf("dynfield: offset=%d, offset2=%d, offset3=%d\n",
> > + offset, offset2, offset3);
> > +
> > + ret = rte_mbuf_dynfield_register(&dynfield_fail_big);
> > + if (ret != -1)
> > + GOTO_FAIL("dynamic field creation should fail (too big)");
> > +
> > + ret = rte_mbuf_dynfield_register(&dynfield_fail_align);
> > + if (ret != -1)
> > + GOTO_FAIL("dynamic field creation should fail (bad alignment)");
> > +
> > + ret = rte_mbuf_dynfield_register_offset(&dynfield_fail_align,
> > + offsetof(struct rte_mbuf, ol_flags));
> > + if (ret != -1)
> > + GOTO_FAIL("dynamic field creation should fail (not avail)");
> > +
> > + flag = rte_mbuf_dynflag_register(&dynflag);
> > + if (flag == -1)
> > + GOTO_FAIL("failed to register dynamic flag, flag=%d: %s",
> > + flag, strerror(errno));
> > +
> > + ret = rte_mbuf_dynflag_register(&dynflag);
> > + if (ret != flag)
> > + GOTO_FAIL("failed to lookup dynamic flag, ret=%d: %s",
> > + ret, strerror(errno));
> > +
> > + flag2 = rte_mbuf_dynflag_register(&dynflag2);
> > + if (flag2 == -1 || flag2 == flag)
> > + GOTO_FAIL("failed to register dynamic flag 2, flag2=%d: %s",
> > + flag2, strerror(errno));
> > +
> > + flag3 = rte_mbuf_dynflag_register_bitnum(&dynflag3,
> > + rte_bsf64(PKT_LAST_FREE));
> > + if (flag3 != rte_bsf64(PKT_LAST_FREE))
> > + GOTO_FAIL("failed to register dynamic flag 3, flag2=%d: %s",
> > + flag3, strerror(errno));
> > +
> > + printf("dynflag: flag=%d, flag2=%d, flag3=%d\n", flag, flag2, flag3);
> > +
> > + /* set, get dynamic field */
> > + m = rte_pktmbuf_alloc(pktmbuf_pool);
> > + if (m == NULL)
> > + GOTO_FAIL("Cannot allocate mbuf");
> > +
> > + *RTE_MBUF_DYNFIELD(m, offset, uint8_t *) = 1;
> > + if (*RTE_MBUF_DYNFIELD(m, offset, uint8_t *) != 1)
> > + GOTO_FAIL("failed to read dynamic field");
> > + *RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) = 1000;
> > + if (*RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) != 1000)
> > + GOTO_FAIL("failed to read dynamic field");
> > +
> > + /* set a dynamic flag */
> > + m->ol_flags |= (1ULL << flag);
> > +
> > + rte_mbuf_dyn_dump(stdout);
> > + rte_pktmbuf_free(m);
> > + return 0;
> > +fail:
> > + rte_pktmbuf_free(m);
> > + return -1;
> > +}
> > +#undef GOTO_FAIL
> > +
> > static int
> > test_mbuf(void)
> > {
> > @@ -1295,6 +1432,12 @@ test_mbuf(void)
> > goto err;
> > }
> >
> > + /* test registration of dynamic fields and flags */
> > + if (test_mbuf_dyn(pktmbuf_pool) < 0) {
> > + printf("mbuf dynflag test failed\n");
> > + goto err;
> > + }
> > +
> > /* create a specific pktmbuf pool with a priv_size != 0 and no data
> > * room size */
> > pktmbuf_pool2 = rte_pktmbuf_pool_create("test_pktmbuf_pool2",
> > diff --git a/doc/guides/rel_notes/release_19_11.rst b/doc/guides/rel_notes/release_19_11.rst
> > index 85953b962..9e9c94554 100644
> > --- a/doc/guides/rel_notes/release_19_11.rst
> > +++ b/doc/guides/rel_notes/release_19_11.rst
> > @@ -21,6 +21,13 @@ DPDK Release 19.11
> >
> > xdg-open build/doc/html/guides/rel_notes/release_19_11.html
> >
> > +* **Add support of support dynamic fields and flags in mbuf.**
> > +
> > + This new feature adds the ability to dynamically register some room
> > + for a field or a flag in the mbuf structure. This is typically used
> > + for specific offload features, where adding a static field or flag
> > + in the mbuf is not justified.
> > +
> >
> > New Features
> > ------------
> > diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile
> > index c8f6d2689..5a9bcee73 100644
> > --- a/lib/librte_mbuf/Makefile
> > +++ b/lib/librte_mbuf/Makefile
> > @@ -17,8 +17,10 @@ LIBABIVER := 5
> >
> > # all source are stored in SRCS-y
> > SRCS-$(CONFIG_RTE_LIBRTE_MBUF) := rte_mbuf.c rte_mbuf_ptype.c rte_mbuf_pool_ops.c
> > +SRCS-$(CONFIG_RTE_LIBRTE_MBUF) += rte_mbuf_dyn.c
> >
> > # install includes
> > SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include := rte_mbuf.h rte_mbuf_ptype.h rte_mbuf_pool_ops.h
> > +SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include += rte_mbuf_dyn.h
> >
> > include $(RTE_SDK)/mk/rte.lib.mk
> > diff --git a/lib/librte_mbuf/meson.build b/lib/librte_mbuf/meson.build
> > index 6cc11ebb4..9137e8f26 100644
> > --- a/lib/librte_mbuf/meson.build
> > +++ b/lib/librte_mbuf/meson.build
> > @@ -2,8 +2,10 @@
> > # Copyright(c) 2017 Intel Corporation
> >
> > version = 5
> > -sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c')
> > -headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h')
> > +sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c',
> > + 'rte_mbuf_dyn.c')
> > +headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h',
> > + 'rte_mbuf_dyn.h')
> > deps += ['mempool']
> >
> > allow_experimental_apis = true
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
> > index fb0849ac1..5740b1e93 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -198,9 +198,12 @@ extern "C" {
> > #define PKT_RX_OUTER_L4_CKSUM_GOOD (1ULL << 22)
> > #define PKT_RX_OUTER_L4_CKSUM_INVALID ((1ULL << 21) | (1ULL << 22))
> >
> > -/* add new RX flags here */
> > +/* add new RX flags here, don't forget to update PKT_FIRST_FREE */
> >
> > -/* add new TX flags here */
> > +#define PKT_FIRST_FREE (1ULL << 23)
> > +#define PKT_LAST_FREE (1ULL << 39)
> > +
> > +/* add new TX flags here, don't forget to update PKT_LAST_FREE */
> >
> > /**
> > * Indicate that the metadata field in the mbuf is in use.
> > @@ -738,6 +741,7 @@ struct rte_mbuf {
> > */
> > struct rte_mbuf_ext_shared_info *shinfo;
> >
> > + uint64_t dynfield1[2]; /**< Reserved for dynamic fields. */
> > } __rte_cache_aligned;
> >
> > /**
> > @@ -1684,6 +1688,20 @@ rte_pktmbuf_attach_extbuf(struct rte_mbuf *m, void *buf_addr,
> > */
> > #define rte_pktmbuf_detach_extbuf(m) rte_pktmbuf_detach(m)
> >
> > +/**
> > + * Copy dynamic fields from m_src to m_dst.
> > + *
> > + * @param m_dst
> > + * The destination mbuf.
> > + * @param m_src
> > + * The source mbuf.
> > + */
> > +static inline void
> > +rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> > +{
> > + memcpy(&mdst->dynfield1, msrc->dynfield1, sizeof(mdst->dynfield1));
> > +}
> > +
> > /* internal */
> > static inline void
> > __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> > @@ -1695,6 +1713,7 @@ __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> > mdst->hash = msrc->hash;
> > mdst->packet_type = msrc->packet_type;
> > mdst->timestamp = msrc->timestamp;
> > + rte_mbuf_dynfield_copy(mdst, msrc);
> > }
> >
> > /**
> > diff --git a/lib/librte_mbuf/rte_mbuf_dyn.c b/lib/librte_mbuf/rte_mbuf_dyn.c
> > new file mode 100644
> > index 000000000..9ef235483
> > --- /dev/null
> > +++ b/lib/librte_mbuf/rte_mbuf_dyn.c
> > @@ -0,0 +1,548 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright 2019 6WIND S.A.
> > + */
> > +
> > +#include <sys/queue.h>
> > +#include <stdint.h>
> > +#include <limits.h>
> > +
> > +#include <rte_common.h>
> > +#include <rte_eal.h>
> > +#include <rte_eal_memconfig.h>
> > +#include <rte_tailq.h>
> > +#include <rte_errno.h>
> > +#include <rte_malloc.h>
> > +#include <rte_string_fns.h>
> > +#include <rte_mbuf.h>
> > +#include <rte_mbuf_dyn.h>
> > +
> > +#define RTE_MBUF_DYN_MZNAME "rte_mbuf_dyn"
> > +
> > +struct mbuf_dynfield_elt {
> > + TAILQ_ENTRY(mbuf_dynfield_elt) next;
> > + struct rte_mbuf_dynfield params;
> > + size_t offset;
> > +};
> > +TAILQ_HEAD(mbuf_dynfield_list, rte_tailq_entry);
> > +
> > +static struct rte_tailq_elem mbuf_dynfield_tailq = {
> > + .name = "RTE_MBUF_DYNFIELD",
> > +};
> > +EAL_REGISTER_TAILQ(mbuf_dynfield_tailq);
> > +
> > +struct mbuf_dynflag_elt {
> > + TAILQ_ENTRY(mbuf_dynflag_elt) next;
> > + struct rte_mbuf_dynflag params;
> > + unsigned int bitnum;
> > +};
> > +TAILQ_HEAD(mbuf_dynflag_list, rte_tailq_entry);
> > +
> > +static struct rte_tailq_elem mbuf_dynflag_tailq = {
> > + .name = "RTE_MBUF_DYNFLAG",
> > +};
> > +EAL_REGISTER_TAILQ(mbuf_dynflag_tailq);
> > +
> > +struct mbuf_dyn_shm {
> > + /**
> > + * For each mbuf byte, free_space[i] != 0 if space is free.
> > + * The value is the size of the biggest aligned element that
> > + * can fit in the zone.
> > + */
> > + uint8_t free_space[sizeof(struct rte_mbuf)];
> > + /** Bitfield of available flags. */
> > + uint64_t free_flags;
> > +};
> > +static struct mbuf_dyn_shm *shm;
> > +
> > +/* Set the value of free_space[] according to the size and alignment of
> > + * the free areas. This helps to select the best place when reserving a
> > + * dynamic field. Assume tailq is locked.
> > + */
> > +static void
> > +process_score(void)
> > +{
> > + size_t off, align, size, i;
> > +
> > + /* first, erase previous info */
> > + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> > + if (shm->free_space[i])
> > + shm->free_space[i] = 1;
> > + }
> > +
> > + for (off = 0; off < sizeof(struct rte_mbuf); off++) {
> > + /* get the size of the free zone */
> > + for (size = 0; shm->free_space[off + size]; size++)
> > + ;
> > + if (size == 0)
> > + continue;
> > +
> > + /* get the alignment of biggest object that can fit in
> > + * the zone at this offset.
> > + */
> > + for (align = 1;
> > + (off % (align << 1)) == 0 && (align << 1) <= size;
> > + align <<= 1)
> > + ;
> > +
> > + /* save it in free_space[] */
> > + for (i = off; i < off + size; i++)
> > + shm->free_space[i] = RTE_MAX(align, shm->free_space[i]);
> > + }
> > +}
> > +
> > +/* Allocate and initialize the shared memory. Assume tailq is locked */
> > +static int
> > +init_shared_mem(void)
> > +{
> > + const struct rte_memzone *mz;
> > + uint64_t mask;
> > +
> > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> > + mz = rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
> > + sizeof(struct mbuf_dyn_shm),
> > + SOCKET_ID_ANY, 0,
> > + RTE_CACHE_LINE_SIZE);
> > + } else {
> > + mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
> > + }
> > + if (mz == NULL)
> > + return -1;
> > +
> > + shm = mz->addr;
> > +
> > +#define mark_free(field) \
> > + memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
> > + 1, sizeof(((struct rte_mbuf *)0)->field))
>
> Still think it would look nicer without multi-line macro defines/undef in the middle of the function.
I rather think that macro helps to make the code more readable, but it's
probably just a matter of taste. Will someone puts a contract on me if I
keep it like this? If yes I'll do the change ;)
> > +
> > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> > + /* init free_space, keep it sync'd with
> > + * rte_mbuf_dynfield_copy().
> > + */
> > + memset(shm, 0, sizeof(*shm));
> > + mark_free(dynfield1);
> > +
> > + /* init free_flags */
> > + for (mask = PKT_FIRST_FREE; mask <= PKT_LAST_FREE; mask <<= 1)
> > + shm->free_flags |= mask;
> > +
> > + process_score();
> > + }
> > +#undef mark_free
> > +
> > + return 0;
> > +}
> > +
> > +/* check if this offset can be used */
> > +static int
> > +check_offset(size_t offset, size_t size, size_t align)
> > +{
> > + size_t i;
> > +
> > + if ((offset & (align - 1)) != 0)
> > + return -1;
> > + if (offset + size > sizeof(struct rte_mbuf))
> > + return -1;
> > +
> > + for (i = 0; i < size; i++) {
> > + if (!shm->free_space[i + offset])
> > + return -1;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +/* assume tailq is locked */
> > +static struct mbuf_dynfield_elt *
> > +__mbuf_dynfield_lookup(const char *name)
> > +{
> > + struct mbuf_dynfield_list *mbuf_dynfield_list;
> > + struct mbuf_dynfield_elt *mbuf_dynfield;
> > + struct rte_tailq_entry *te;
> > +
> > + mbuf_dynfield_list = RTE_TAILQ_CAST(
> > + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> > +
> > + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> > + mbuf_dynfield = (struct mbuf_dynfield_elt *)te->data;
> > + if (strcmp(name, mbuf_dynfield->params.name) == 0)
> > + break;
> > + }
> > +
> > + if (te == NULL) {
> > + rte_errno = ENOENT;
> > + return NULL;
> > + }
> > +
> > + return mbuf_dynfield;
> > +}
> > +
> > +int
> > +rte_mbuf_dynfield_lookup(const char *name, struct rte_mbuf_dynfield *params)
> > +{
> > + struct mbuf_dynfield_elt *mbuf_dynfield;
> > +
> > + if (shm == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_read_lock();
> > + mbuf_dynfield = __mbuf_dynfield_lookup(name);
> > + rte_mcfg_tailq_read_unlock();
> > +
> > + if (mbuf_dynfield == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + if (params != NULL)
> > + memcpy(params, &mbuf_dynfield->params, sizeof(*params));
> > +
> > + return mbuf_dynfield->offset;
> > +}
> > +
> > +static int mbuf_dynfield_cmp(const struct rte_mbuf_dynfield *params1,
> > + const struct rte_mbuf_dynfield *params2)
> > +{
> > + if (strcmp(params1->name, params2->name))
> > + return -1;
> > + if (params1->size != params2->size)
> > + return -1;
> > + if (params1->align != params2->align)
> > + return -1;
> > + if (params1->flags != params2->flags)
> > + return -1;
> > + return 0;
> > +}
> > +
> > +/* assume tailq is locked */
> > +static int
> > +__rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params,
> > + size_t req)
> > +{
> > + struct mbuf_dynfield_list *mbuf_dynfield_list;
> > + struct mbuf_dynfield_elt *mbuf_dynfield = NULL;
> > + struct rte_tailq_entry *te = NULL;
> > + unsigned int best_zone = UINT_MAX;
> > + size_t i, offset;
> > + int ret;
> > +
> > + if (shm == NULL && init_shared_mem() < 0)
> > + return -1;
> > +
> > + mbuf_dynfield = __mbuf_dynfield_lookup(params->name);
> > + if (mbuf_dynfield != NULL) {
> > + if (req != SIZE_MAX && req != mbuf_dynfield->offset) {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + if (mbuf_dynfield_cmp(params, &mbuf_dynfield->params) < 0) {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + return mbuf_dynfield->offset;
> > + }
> > +
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> > + rte_errno = EPERM;
> > + return -1;
> > + }
> > +
> > + if (req == SIZE_MAX) {
> > + for (offset = 0;
> > + offset < sizeof(struct rte_mbuf);
> > + offset++) {
> > + if (check_offset(offset, params->size,
> > + params->align) == 0 &&
> > + shm->free_space[offset] < best_zone) {
>
> Probably worth to explain a bit more here about best_zone logic -
> trying to find offset with minimal score (minimal continuous length), etc.
Yes, will do.
> > + best_zone = shm->free_space[offset];
> > + req = offset;
> > + }
> > + }
> > + if (req == SIZE_MAX) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > + } else {
> > + if (check_offset(req, params->size, params->align) < 0) {
> > + rte_errno = EBUSY;
> > + return -1;
> > + }
> > + }
> > +
> > + offset = req;
> > + mbuf_dynfield_list = RTE_TAILQ_CAST(
> > + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> > +
> > + te = rte_zmalloc("MBUF_DYNFIELD_TAILQ_ENTRY", sizeof(*te), 0);
> > + if (te == NULL)
> > + return -1;
> > +
> > + mbuf_dynfield = rte_zmalloc("mbuf_dynfield", sizeof(*mbuf_dynfield), 0);
> > + if (mbuf_dynfield == NULL) {
> > + rte_free(te);
> > + return -1;
> > + }
> > +
> > + ret = strlcpy(mbuf_dynfield->params.name, params->name,
> > + sizeof(mbuf_dynfield->params.name));
> > + if (ret < 0 || ret >= (int)sizeof(mbuf_dynfield->params.name)) {
> > + rte_errno = ENAMETOOLONG;
> > + rte_free(mbuf_dynfield);
> > + rte_free(te);
> > + return -1;
> > + }
> > + memcpy(&mbuf_dynfield->params, params, sizeof(mbuf_dynfield->params));
> > + mbuf_dynfield->offset = offset;
> > + te->data = mbuf_dynfield;
> > +
> > + TAILQ_INSERT_TAIL(mbuf_dynfield_list, te, next);
> > +
> > + for (i = offset; i < offset + params->size; i++)
> > + shm->free_space[i] = 0;
> > + process_score();
> > +
> > + RTE_LOG(DEBUG, MBUF, "Registered dynamic field %s (sz=%zu, al=%zu, fl=0x%x) -> %zd\n",
> > + params->name, params->size, params->align, params->flags,
> > + offset);
> > +
> > + return offset;
> > +}
> > +
> > +int
> > +rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params,
> > + size_t req)
> > +{
> > + int ret;
> > +
> > + if (params->size >= sizeof(struct rte_mbuf)) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > + if (!rte_is_power_of_2(params->align)) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > + if (params->flags != 0) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_write_lock();
> > + ret = __rte_mbuf_dynfield_register_offset(params, req);
> > + rte_mcfg_tailq_write_unlock();
> > +
> > + return ret;
> > +}
> > +
> > +int
> > +rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params)
> > +{
> > + return rte_mbuf_dynfield_register_offset(params, SIZE_MAX);
> > +}
> > +
> > +/* assume tailq is locked */
> > +static struct mbuf_dynflag_elt *
> > +__mbuf_dynflag_lookup(const char *name)
> > +{
> > + struct mbuf_dynflag_list *mbuf_dynflag_list;
> > + struct mbuf_dynflag_elt *mbuf_dynflag;
> > + struct rte_tailq_entry *te;
> > +
> > + mbuf_dynflag_list = RTE_TAILQ_CAST(
> > + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> > +
> > + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> > + mbuf_dynflag = (struct mbuf_dynflag_elt *)te->data;
> > + if (strncmp(name, mbuf_dynflag->params.name,
> > + RTE_MBUF_DYN_NAMESIZE) == 0)
> > + break;
> > + }
> > +
> > + if (te == NULL) {
> > + rte_errno = ENOENT;
> > + return NULL;
> > + }
> > +
> > + return mbuf_dynflag;
> > +}
> > +
> > +int
> > +rte_mbuf_dynflag_lookup(const char *name,
> > + struct rte_mbuf_dynflag *params)
> > +{
> > + struct mbuf_dynflag_elt *mbuf_dynflag;
> > +
> > + if (shm == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_read_lock();
> > + mbuf_dynflag = __mbuf_dynflag_lookup(name);
> > + rte_mcfg_tailq_read_unlock();
> > +
> > + if (mbuf_dynflag == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + if (params != NULL)
> > + memcpy(params, &mbuf_dynflag->params, sizeof(*params));
> > +
> > + return mbuf_dynflag->bitnum;
> > +}
> > +
> > +static int mbuf_dynflag_cmp(const struct rte_mbuf_dynflag *params1,
> > + const struct rte_mbuf_dynflag *params2)
> > +{
> > + if (strcmp(params1->name, params2->name))
> > + return -1;
> > + if (params1->flags != params2->flags)
> > + return -1;
> > + return 0;
> > +}
> > +
> > +/* assume tailq is locked */
> > +static int
> > +__rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > + unsigned int req)
> > +{
> > + struct mbuf_dynflag_list *mbuf_dynflag_list;
> > + struct mbuf_dynflag_elt *mbuf_dynflag = NULL;
> > + struct rte_tailq_entry *te = NULL;
> > + unsigned int bitnum;
> > + int ret;
> > +
> > + if (shm == NULL && init_shared_mem() < 0)
> > + return -1;
> > +
> > + mbuf_dynflag = __mbuf_dynflag_lookup(params->name);
> > + if (mbuf_dynflag != NULL) {
> > + if (req != UINT_MAX && req != mbuf_dynflag->bitnum) {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + if (mbuf_dynflag_cmp(params, &mbuf_dynflag->params) < 0) {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + return mbuf_dynflag->bitnum;
> > + }
> > +
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> > + rte_errno = EPERM;
> > + return -1;
> > + }
> > +
> > + if (req == UINT_MAX) {
> > + if (shm->free_flags == 0) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > + bitnum = rte_bsf64(shm->free_flags);
> > + } else {
> > + if ((shm->free_flags & (1ULL << req)) == 0) {
> > + rte_errno = EBUSY;
> > + return -1;
> > + }
> > + bitnum = req;
> > + }
> > +
> > + mbuf_dynflag_list = RTE_TAILQ_CAST(
> > + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> > +
> > + te = rte_zmalloc("MBUF_DYNFLAG_TAILQ_ENTRY", sizeof(*te), 0);
> > + if (te == NULL)
> > + return -1;
> > +
> > + mbuf_dynflag = rte_zmalloc("mbuf_dynflag", sizeof(*mbuf_dynflag), 0);
> > + if (mbuf_dynflag == NULL) {
> > + rte_free(te);
> > + return -1;
> > + }
> > +
> > + ret = strlcpy(mbuf_dynflag->params.name, params->name,
> > + sizeof(mbuf_dynflag->params.name));
> > + if (ret < 0 || ret >= (int)sizeof(mbuf_dynflag->params.name)) {
> > + rte_free(mbuf_dynflag);
> > + rte_free(te);
> > + rte_errno = ENAMETOOLONG;
> > + return -1;
> > + }
> > + mbuf_dynflag->bitnum = bitnum;
> > + te->data = mbuf_dynflag;
> > +
> > + TAILQ_INSERT_TAIL(mbuf_dynflag_list, te, next);
> > +
> > + shm->free_flags &= ~(1ULL << bitnum);
> > +
> > + RTE_LOG(DEBUG, MBUF, "Registered dynamic flag %s (fl=0x%x) -> %u\n",
> > + params->name, params->flags, bitnum);
> > +
> > + return bitnum;
> > +}
> > +
> > +int
> > +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > + unsigned int req)
> > +{
> > + int ret;
> > +
> > + if (req != UINT_MAX && req >= 64) {
>
> Might be better to replace 64 with something like sizeof(mbuf->ol_flags) * CHAR_BIT or so.
Will do.
> Apart from that:
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Thanks for the review
Olivier
On Wed, Oct 23, 2019 at 03:16:13AM +0000, Wang, Haiyue wrote:
> > -----Original Message-----
> > From: Ananyev, Konstantin
> > Sent: Wednesday, October 23, 2019 06:52
> > To: Olivier Matz <olivier.matz@6wind.com>; dev@dpdk.org
> > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>; Wang,
> > Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > <keith.wiles@intel.com>; Morten Brørup <mb@smartsharesystems.com>; Stephen Hemminger
> > <stephen@networkplumber.org>; Thomas Monjalon <thomas@monjalon.net>
> > Subject: RE: [PATCH v2] mbuf: support dynamic fields and flags
> >
> >
> > > Many features require to store data inside the mbuf. As the room in mbuf
> > > structure is limited, it is not possible to have a field for each
> > > feature. Also, changing fields in the mbuf structure can break the API
> > > or ABI.
> > >
> > > This commit addresses these issues, by enabling the dynamic registration
> > > of fields or flags:
> > >
> > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > given size (>= 1 byte) and alignment constraint.
> > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > >
> > > The typical use case is a PMD that registers space for an offload
> > > feature, when the application requests to enable this feature. As
> > > the space in mbuf is limited, the space should only be reserved if it
> > > is going to be used (i.e when the application explicitly asks for it).
> > >
> > > The registration can be done at any moment, but it is not possible
> > > to unregister fields or flags for now.
> > >
> > > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > > ---
> > >
> > > v2
> > >
> > > * Rebase on top of master: solve conflict with Stephen's patchset
> > > (packet copy)
> > > * Add new apis to register a dynamic field/flag at a specific place
> > > * Add a dump function (sugg by David)
> > > * Enhance field registration function to select the best offset, keeping
> > > large aligned zones as much as possible (sugg by Konstantin)
> > > * Use a size_t and unsigned int instead of int when relevant
> > > (sugg by Konstantin)
> > > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > > (sugg by Konstantin)
> > > * Remove unused argument in private function (sugg by Konstantin)
> > > * Fix and simplify locking (sugg by Konstantin)
> > > * Fix minor typo
> > >
> > > rfc -> v1
> > >
> > > * Rebase on top of master
> > > * Change registration API to use a structure instead of
> > > variables, getting rid of #defines (Stephen's comment)
> > > * Update flag registration to use a similar API as fields.
> > > * Change max name length from 32 to 64 (sugg. by Thomas)
> > > * Enhance API documentation (Haiyue's and Andrew's comments)
> > > * Add a debug log at registration
> > > * Add some words in release note
> > > * Did some performance tests (sugg. by Andrew):
> > > On my platform, reading a dynamic field takes ~3 cycles more
> > > than a static field, and ~2 cycles more for writing.
> > >
> > > app/test/test_mbuf.c | 145 ++++++-
> > > doc/guides/rel_notes/release_19_11.rst | 7 +
> > > lib/librte_mbuf/Makefile | 2 +
> > > lib/librte_mbuf/meson.build | 6 +-
> > > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > > 8 files changed, 959 insertions(+), 5 deletions(-)
> > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> > >
> > > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > > index b9c2b2500..01cafad59 100644
> > > --- a/app/test/test_mbuf.c
> > > +++ b/app/test/test_mbuf.c
> > > @@ -28,6 +28,7 @@
> > > #include <rte_random.h>
> > > #include <rte_cycles.h>
> > > #include <rte_malloc.h>
> > > +#include <rte_mbuf_dyn.h>
> > >
>
> [snip]
> > > +int
> > > +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > > + unsigned int req)
> > > +{
> > > + int ret;
> > > +
> > > + if (req != UINT_MAX && req >= 64) {
> >
> > Might be better to replace 64 with something like sizeof(mbuf->ol_flags) * CHAR_BIT or so.
>
> Might introduce a new macro like kernel:
>
> /**
> * FIELD_SIZEOF - get the size of a struct's field
> * @t: the target struct
> * @f: the target struct's field
> * Return: the size of @f in the struct definition without having a
> * declared instance of @t.
> */
> #define FIELD_SIZEOF(t, f) (sizeof(((t*)0)->f))
>
> Then: FIELD_SIZEOF(rte_mbuf, ol_flags) * CHAR_BIT
Good idea, thanks
On Wed, Oct 23, 2019 at 12:19:46PM +0200, Olivier Matz wrote:
> On Tue, Oct 22, 2019 at 10:51:51PM +0000, Ananyev, Konstantin wrote:
(...)
> > > +/* Allocate and initialize the shared memory. Assume tailq is locked */
> > > +static int
> > > +init_shared_mem(void)
> > > +{
> > > + const struct rte_memzone *mz;
> > > + uint64_t mask;
> > > +
> > > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> > > + mz = rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
> > > + sizeof(struct mbuf_dyn_shm),
> > > + SOCKET_ID_ANY, 0,
> > > + RTE_CACHE_LINE_SIZE);
> > > + } else {
> > > + mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
> > > + }
> > > + if (mz == NULL)
> > > + return -1;
> > > +
> > > + shm = mz->addr;
> > > +
> > > +#define mark_free(field) \
> > > + memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
> > > + 1, sizeof(((struct rte_mbuf *)0)->field))
> >
> > Still think it would look nicer without multi-line macro defines/undef in the middle of the function.
>
> I rather think that macro helps to make the code more readable, but it's
> probably just a matter of taste. Will someone puts a contract on me if I
> keep it like this? If yes I'll do the change ;)
More seriously, do you prefer if I move the macro definition above the
function?
> -----Original Message-----
> From: Olivier Matz <olivier.matz@6wind.com>
> Sent: Wednesday, October 23, 2019 12:46 PM
> To: Ananyev, Konstantin <konstantin.ananyev@intel.com>
> Cc: dev@dpdk.org; Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>; Wang,
> Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith <keith.wiles@intel.com>; Morten
> Brørup <mb@smartsharesystems.com>; Stephen Hemminger <stephen@networkplumber.org>; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: Re: [PATCH v2] mbuf: support dynamic fields and flags
>
> On Wed, Oct 23, 2019 at 12:19:46PM +0200, Olivier Matz wrote:
> > On Tue, Oct 22, 2019 at 10:51:51PM +0000, Ananyev, Konstantin wrote:
>
> (...)
>
> > > > +/* Allocate and initialize the shared memory. Assume tailq is locked */
> > > > +static int
> > > > +init_shared_mem(void)
> > > > +{
> > > > + const struct rte_memzone *mz;
> > > > + uint64_t mask;
> > > > +
> > > > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> > > > + mz = rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
> > > > + sizeof(struct mbuf_dyn_shm),
> > > > + SOCKET_ID_ANY, 0,
> > > > + RTE_CACHE_LINE_SIZE);
> > > > + } else {
> > > > + mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
> > > > + }
> > > > + if (mz == NULL)
> > > > + return -1;
> > > > +
> > > > + shm = mz->addr;
> > > > +
> > > > +#define mark_free(field) \
> > > > + memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
> > > > + 1, sizeof(((struct rte_mbuf *)0)->field))
> > >
> > > Still think it would look nicer without multi-line macro defines/undef in the middle of the function.
> >
> > I rather think that macro helps to make the code more readable, but it's
> > probably just a matter of taste. Will someone puts a contract on me if I
> > keep it like this? If yes I'll do the change ;)
>
> More seriously, do you prefer if I move the macro definition above the
> function?
Yes, would look better to me.
Hi Olivier,
Thursday, October 17, 2019 5:42 PM, Olivier Matz:
> Subject: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and flags
>
> Many features require to store data inside the mbuf. As the room in mbuf
> structure is limited, it is not possible to have a field for each feature. Also,
> changing fields in the mbuf structure can break the API or ABI.
>
> This commit addresses these issues, by enabling the dynamic registration of
> fields or flags:
>
> - a dynamic field is a named area in the rte_mbuf structure, with a
> given size (>= 1 byte) and alignment constraint.
> - a dynamic flag is a named bit in the rte_mbuf structure.
>
> The typical use case is a PMD that registers space for an offload feature,
> when the application requests to enable this feature. As the space in mbuf is
> limited, the space should only be reserved if it is going to be used (i.e when
> the application explicitly asks for it).
According to description, the dynamic field enables custom application and supported PMDs to use the dynamic part of the mbuf for their specific needs.
However the mechanism to report and activate the field/flag registration comes from the general OFFLOAD flags.
Maybe it will be better to an option to query and select dynamic fields for PMD outside of the standard ethdev offload flags?
>
> The registration can be done at any moment, but it is not possible to
> unregister fields or flags for now.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>
> v2
>
> * Rebase on top of master: solve conflict with Stephen's patchset
> (packet copy)
> * Add new apis to register a dynamic field/flag at a specific place
> * Add a dump function (sugg by David)
> * Enhance field registration function to select the best offset, keeping
> large aligned zones as much as possible (sugg by Konstantin)
> * Use a size_t and unsigned int instead of int when relevant
> (sugg by Konstantin)
> * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> (sugg by Konstantin)
> * Remove unused argument in private function (sugg by Konstantin)
> * Fix and simplify locking (sugg by Konstantin)
> * Fix minor typo
>
> rfc -> v1
>
> * Rebase on top of master
> * Change registration API to use a structure instead of
> variables, getting rid of #defines (Stephen's comment)
> * Update flag registration to use a similar API as fields.
> * Change max name length from 32 to 64 (sugg. by Thomas)
> * Enhance API documentation (Haiyue's and Andrew's comments)
> * Add a debug log at registration
> * Add some words in release note
> * Did some performance tests (sugg. by Andrew):
> On my platform, reading a dynamic field takes ~3 cycles more
> than a static field, and ~2 cycles more for writing.
>
> app/test/test_mbuf.c | 145 ++++++-
> doc/guides/rel_notes/release_19_11.rst | 7 +
> lib/librte_mbuf/Makefile | 2 +
> lib/librte_mbuf/meson.build | 6 +-
> lib/librte_mbuf/rte_mbuf.h | 23 +-
> lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> lib/librte_mbuf/rte_mbuf_version.map | 7 +
> 8 files changed, 959 insertions(+), 5 deletions(-) create mode 100644
> lib/librte_mbuf/rte_mbuf_dyn.c create mode 100644
> lib/librte_mbuf/rte_mbuf_dyn.h
>
> diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c index
> b9c2b2500..01cafad59 100644
> --- a/app/test/test_mbuf.c
> +++ b/app/test/test_mbuf.c
> @@ -28,6 +28,7 @@
> #include <rte_random.h>
> #include <rte_cycles.h>
> #include <rte_malloc.h>
> +#include <rte_mbuf_dyn.h>
>
> #include "test.h"
>
> @@ -657,7 +658,6 @@ test_attach_from_different_pool(struct
> rte_mempool *pktmbuf_pool,
> rte_pktmbuf_free(clone2);
> return -1;
> }
> -#undef GOTO_FAIL
>
> /*
> * test allocation and free of mbufs
> @@ -1276,6 +1276,143 @@ test_tx_offload(void)
> return (v1 == v2) ? 0 : -EINVAL;
> }
>
> +static int
> +test_mbuf_dyn(struct rte_mempool *pktmbuf_pool) {
> + const struct rte_mbuf_dynfield dynfield = {
> + .name = "test-dynfield",
> + .size = sizeof(uint8_t),
> + .align = __alignof__(uint8_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield2 = {
> + .name = "test-dynfield2",
> + .size = sizeof(uint16_t),
> + .align = __alignof__(uint16_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield3 = {
> + .name = "test-dynfield3",
> + .size = sizeof(uint8_t),
> + .align = __alignof__(uint8_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield_fail_big = {
> + .name = "test-dynfield-fail-big",
> + .size = 256,
> + .align = 1,
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield_fail_align = {
> + .name = "test-dynfield-fail-align",
> + .size = 1,
> + .align = 3,
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag = {
> + .name = "test-dynflag",
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag2 = {
> + .name = "test-dynflag2",
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag3 = {
> + .name = "test-dynflag3",
> + .flags = 0,
> + };
> + struct rte_mbuf *m = NULL;
> + int offset, offset2, offset3;
> + int flag, flag2, flag3;
> + int ret;
> +
> + printf("Test mbuf dynamic fields and flags\n");
> + rte_mbuf_dyn_dump(stdout);
> +
> + offset = rte_mbuf_dynfield_register(&dynfield);
> + if (offset == -1)
> + GOTO_FAIL("failed to register dynamic field, offset=%d: %s",
> + offset, strerror(errno));
> +
> + ret = rte_mbuf_dynfield_register(&dynfield);
> + if (ret != offset)
> + GOTO_FAIL("failed to lookup dynamic field, ret=%d: %s",
> + ret, strerror(errno));
> +
> + offset2 = rte_mbuf_dynfield_register(&dynfield2);
> + if (offset2 == -1 || offset2 == offset || (offset2 & 1))
> + GOTO_FAIL("failed to register dynamic field 2, offset2=%d:
> %s",
> + offset2, strerror(errno));
> +
> + offset3 = rte_mbuf_dynfield_register_offset(&dynfield3,
> + offsetof(struct rte_mbuf, dynfield1[1]));
> + if (offset3 != offsetof(struct rte_mbuf, dynfield1[1]))
> + GOTO_FAIL("failed to register dynamic field 3, offset=%d:
> %s",
> + offset3, strerror(errno));
> +
> + printf("dynfield: offset=%d, offset2=%d, offset3=%d\n",
> + offset, offset2, offset3);
> +
> + ret = rte_mbuf_dynfield_register(&dynfield_fail_big);
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (too big)");
> +
> + ret = rte_mbuf_dynfield_register(&dynfield_fail_align);
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (bad
> alignment)");
> +
> + ret = rte_mbuf_dynfield_register_offset(&dynfield_fail_align,
> + offsetof(struct rte_mbuf, ol_flags));
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (not avail)");
> +
> + flag = rte_mbuf_dynflag_register(&dynflag);
> + if (flag == -1)
> + GOTO_FAIL("failed to register dynamic flag, flag=%d: %s",
> + flag, strerror(errno));
> +
> + ret = rte_mbuf_dynflag_register(&dynflag);
> + if (ret != flag)
> + GOTO_FAIL("failed to lookup dynamic flag, ret=%d: %s",
> + ret, strerror(errno));
> +
> + flag2 = rte_mbuf_dynflag_register(&dynflag2);
> + if (flag2 == -1 || flag2 == flag)
> + GOTO_FAIL("failed to register dynamic flag 2, flag2=%d: %s",
> + flag2, strerror(errno));
> +
> + flag3 = rte_mbuf_dynflag_register_bitnum(&dynflag3,
> + rte_bsf64(PKT_LAST_FREE));
> + if (flag3 != rte_bsf64(PKT_LAST_FREE))
> + GOTO_FAIL("failed to register dynamic flag 3, flag2=%d: %s",
> + flag3, strerror(errno));
> +
> + printf("dynflag: flag=%d, flag2=%d, flag3=%d\n", flag, flag2, flag3);
> +
> + /* set, get dynamic field */
> + m = rte_pktmbuf_alloc(pktmbuf_pool);
> + if (m == NULL)
> + GOTO_FAIL("Cannot allocate mbuf");
> +
> + *RTE_MBUF_DYNFIELD(m, offset, uint8_t *) = 1;
> + if (*RTE_MBUF_DYNFIELD(m, offset, uint8_t *) != 1)
> + GOTO_FAIL("failed to read dynamic field");
> + *RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) = 1000;
> + if (*RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) != 1000)
> + GOTO_FAIL("failed to read dynamic field");
> +
> + /* set a dynamic flag */
> + m->ol_flags |= (1ULL << flag);
> +
> + rte_mbuf_dyn_dump(stdout);
> + rte_pktmbuf_free(m);
> + return 0;
> +fail:
> + rte_pktmbuf_free(m);
> + return -1;
> +}
> +#undef GOTO_FAIL
> +
> static int
> test_mbuf(void)
> {
> @@ -1295,6 +1432,12 @@ test_mbuf(void)
> goto err;
> }
>
> + /* test registration of dynamic fields and flags */
> + if (test_mbuf_dyn(pktmbuf_pool) < 0) {
> + printf("mbuf dynflag test failed\n");
> + goto err;
> + }
> +
> /* create a specific pktmbuf pool with a priv_size != 0 and no data
> * room size */
> pktmbuf_pool2 = rte_pktmbuf_pool_create("test_pktmbuf_pool2",
> diff --git a/doc/guides/rel_notes/release_19_11.rst
> b/doc/guides/rel_notes/release_19_11.rst
> index 85953b962..9e9c94554 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -21,6 +21,13 @@ DPDK Release 19.11
>
> xdg-open build/doc/html/guides/rel_notes/release_19_11.html
>
> +* **Add support of support dynamic fields and flags in mbuf.**
> +
> + This new feature adds the ability to dynamically register some room
> + for a field or a flag in the mbuf structure. This is typically used
> + for specific offload features, where adding a static field or flag in
> + the mbuf is not justified.
> +
>
> New Features
> ------------
> diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile index
> c8f6d2689..5a9bcee73 100644
> --- a/lib/librte_mbuf/Makefile
> +++ b/lib/librte_mbuf/Makefile
> @@ -17,8 +17,10 @@ LIBABIVER := 5
>
> # all source are stored in SRCS-y
> SRCS-$(CONFIG_RTE_LIBRTE_MBUF) := rte_mbuf.c rte_mbuf_ptype.c
> rte_mbuf_pool_ops.c
> +SRCS-$(CONFIG_RTE_LIBRTE_MBUF) += rte_mbuf_dyn.c
>
> # install includes
> SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include := rte_mbuf.h
> rte_mbuf_ptype.h rte_mbuf_pool_ops.h
> +SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include += rte_mbuf_dyn.h
>
> include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/lib/librte_mbuf/meson.build b/lib/librte_mbuf/meson.build index
> 6cc11ebb4..9137e8f26 100644
> --- a/lib/librte_mbuf/meson.build
> +++ b/lib/librte_mbuf/meson.build
> @@ -2,8 +2,10 @@
> # Copyright(c) 2017 Intel Corporation
>
> version = 5
> -sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c') -
> headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h')
> +sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c',
> + 'rte_mbuf_dyn.c')
> +headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h',
> + 'rte_mbuf_dyn.h')
> deps += ['mempool']
>
> allow_experimental_apis = true
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index
> fb0849ac1..5740b1e93 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -198,9 +198,12 @@ extern "C" {
> #define PKT_RX_OUTER_L4_CKSUM_GOOD (1ULL << 22)
> #define PKT_RX_OUTER_L4_CKSUM_INVALID ((1ULL << 21) | (1ULL << 22))
>
> -/* add new RX flags here */
> +/* add new RX flags here, don't forget to update PKT_FIRST_FREE */
>
> -/* add new TX flags here */
> +#define PKT_FIRST_FREE (1ULL << 23)
> +#define PKT_LAST_FREE (1ULL << 39)
> +
> +/* add new TX flags here, don't forget to update PKT_LAST_FREE */
>
> /**
> * Indicate that the metadata field in the mbuf is in use.
> @@ -738,6 +741,7 @@ struct rte_mbuf {
> */
> struct rte_mbuf_ext_shared_info *shinfo;
>
> + uint64_t dynfield1[2]; /**< Reserved for dynamic fields. */
> } __rte_cache_aligned;
>
> /**
> @@ -1684,6 +1688,20 @@ rte_pktmbuf_attach_extbuf(struct rte_mbuf *m,
> void *buf_addr,
> */
> #define rte_pktmbuf_detach_extbuf(m) rte_pktmbuf_detach(m)
>
> +/**
> + * Copy dynamic fields from m_src to m_dst.
> + *
> + * @param m_dst
> + * The destination mbuf.
> + * @param m_src
> + * The source mbuf.
> + */
> +static inline void
> +rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf
> +*msrc) {
> + memcpy(&mdst->dynfield1, msrc->dynfield1, sizeof(mdst-
> >dynfield1)); }
> +
> /* internal */
> static inline void
> __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf
> *msrc) @@ -1695,6 +1713,7 @@ __rte_pktmbuf_copy_hdr(struct rte_mbuf
> *mdst, const struct rte_mbuf *msrc)
> mdst->hash = msrc->hash;
> mdst->packet_type = msrc->packet_type;
> mdst->timestamp = msrc->timestamp;
> + rte_mbuf_dynfield_copy(mdst, msrc);
> }
>
> /**
> diff --git a/lib/librte_mbuf/rte_mbuf_dyn.c
> b/lib/librte_mbuf/rte_mbuf_dyn.c new file mode 100644 index
> 000000000..9ef235483
> --- /dev/null
> +++ b/lib/librte_mbuf/rte_mbuf_dyn.c
> @@ -0,0 +1,548 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 6WIND S.A.
> + */
> +
> +#include <sys/queue.h>
> +#include <stdint.h>
> +#include <limits.h>
> +
> +#include <rte_common.h>
> +#include <rte_eal.h>
> +#include <rte_eal_memconfig.h>
> +#include <rte_tailq.h>
> +#include <rte_errno.h>
> +#include <rte_malloc.h>
> +#include <rte_string_fns.h>
> +#include <rte_mbuf.h>
> +#include <rte_mbuf_dyn.h>
> +
> +#define RTE_MBUF_DYN_MZNAME "rte_mbuf_dyn"
> +
> +struct mbuf_dynfield_elt {
> + TAILQ_ENTRY(mbuf_dynfield_elt) next;
> + struct rte_mbuf_dynfield params;
> + size_t offset;
> +};
> +TAILQ_HEAD(mbuf_dynfield_list, rte_tailq_entry);
> +
> +static struct rte_tailq_elem mbuf_dynfield_tailq = {
> + .name = "RTE_MBUF_DYNFIELD",
> +};
> +EAL_REGISTER_TAILQ(mbuf_dynfield_tailq);
> +
> +struct mbuf_dynflag_elt {
> + TAILQ_ENTRY(mbuf_dynflag_elt) next;
> + struct rte_mbuf_dynflag params;
> + unsigned int bitnum;
> +};
> +TAILQ_HEAD(mbuf_dynflag_list, rte_tailq_entry);
> +
> +static struct rte_tailq_elem mbuf_dynflag_tailq = {
> + .name = "RTE_MBUF_DYNFLAG",
> +};
> +EAL_REGISTER_TAILQ(mbuf_dynflag_tailq);
> +
> +struct mbuf_dyn_shm {
> + /**
> + * For each mbuf byte, free_space[i] != 0 if space is free.
> + * The value is the size of the biggest aligned element that
> + * can fit in the zone.
> + */
> + uint8_t free_space[sizeof(struct rte_mbuf)];
> + /** Bitfield of available flags. */
> + uint64_t free_flags;
> +};
> +static struct mbuf_dyn_shm *shm;
> +
> +/* Set the value of free_space[] according to the size and alignment of
> + * the free areas. This helps to select the best place when reserving a
> + * dynamic field. Assume tailq is locked.
> + */
> +static void
> +process_score(void)
> +{
> + size_t off, align, size, i;
> +
> + /* first, erase previous info */
> + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> + if (shm->free_space[i])
> + shm->free_space[i] = 1;
> + }
> +
> + for (off = 0; off < sizeof(struct rte_mbuf); off++) {
> + /* get the size of the free zone */
> + for (size = 0; shm->free_space[off + size]; size++)
> + ;
> + if (size == 0)
> + continue;
> +
> + /* get the alignment of biggest object that can fit in
> + * the zone at this offset.
> + */
> + for (align = 1;
> + (off % (align << 1)) == 0 && (align << 1) <= size;
> + align <<= 1)
> + ;
> +
> + /* save it in free_space[] */
> + for (i = off; i < off + size; i++)
> + shm->free_space[i] = RTE_MAX(align, shm-
> >free_space[i]);
> + }
> +}
> +
> +/* Allocate and initialize the shared memory. Assume tailq is locked */
> +static int
> +init_shared_mem(void)
> +{
> + const struct rte_memzone *mz;
> + uint64_t mask;
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + mz =
> rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
> + sizeof(struct
> mbuf_dyn_shm),
> + SOCKET_ID_ANY, 0,
> + RTE_CACHE_LINE_SIZE);
> + } else {
> + mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
> + }
> + if (mz == NULL)
> + return -1;
> +
> + shm = mz->addr;
> +
> +#define mark_free(field) \
> + memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
> + 1, sizeof(((struct rte_mbuf *)0)->field))
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + /* init free_space, keep it sync'd with
> + * rte_mbuf_dynfield_copy().
> + */
> + memset(shm, 0, sizeof(*shm));
> + mark_free(dynfield1);
> +
> + /* init free_flags */
> + for (mask = PKT_FIRST_FREE; mask <= PKT_LAST_FREE; mask
> <<= 1)
> + shm->free_flags |= mask;
> +
> + process_score();
> + }
> +#undef mark_free
> +
> + return 0;
> +}
> +
> +/* check if this offset can be used */
> +static int
> +check_offset(size_t offset, size_t size, size_t align) {
> + size_t i;
> +
> + if ((offset & (align - 1)) != 0)
> + return -1;
> + if (offset + size > sizeof(struct rte_mbuf))
> + return -1;
> +
> + for (i = 0; i < size; i++) {
> + if (!shm->free_space[i + offset])
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static struct mbuf_dynfield_elt *
> +__mbuf_dynfield_lookup(const char *name) {
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *mbuf_dynfield;
> + struct rte_tailq_entry *te;
> +
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> +
> + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> + mbuf_dynfield = (struct mbuf_dynfield_elt *)te->data;
> + if (strcmp(name, mbuf_dynfield->params.name) == 0)
> + break;
> + }
> +
> + if (te == NULL) {
> + rte_errno = ENOENT;
> + return NULL;
> + }
> +
> + return mbuf_dynfield;
> +}
> +
> +int
> +rte_mbuf_dynfield_lookup(const char *name, struct rte_mbuf_dynfield
> +*params) {
> + struct mbuf_dynfield_elt *mbuf_dynfield;
> +
> + if (shm == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_read_lock();
> + mbuf_dynfield = __mbuf_dynfield_lookup(name);
> + rte_mcfg_tailq_read_unlock();
> +
> + if (mbuf_dynfield == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + if (params != NULL)
> + memcpy(params, &mbuf_dynfield->params,
> sizeof(*params));
> +
> + return mbuf_dynfield->offset;
> +}
> +
> +static int mbuf_dynfield_cmp(const struct rte_mbuf_dynfield *params1,
> + const struct rte_mbuf_dynfield *params2) {
> + if (strcmp(params1->name, params2->name))
> + return -1;
> + if (params1->size != params2->size)
> + return -1;
> + if (params1->align != params2->align)
> + return -1;
> + if (params1->flags != params2->flags)
> + return -1;
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static int
> +__rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> *params,
> + size_t req)
> +{
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *mbuf_dynfield = NULL;
> + struct rte_tailq_entry *te = NULL;
> + unsigned int best_zone = UINT_MAX;
> + size_t i, offset;
> + int ret;
> +
> + if (shm == NULL && init_shared_mem() < 0)
> + return -1;
> +
> + mbuf_dynfield = __mbuf_dynfield_lookup(params->name);
> + if (mbuf_dynfield != NULL) {
> + if (req != SIZE_MAX && req != mbuf_dynfield->offset) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + if (mbuf_dynfield_cmp(params, &mbuf_dynfield->params) <
> 0) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + return mbuf_dynfield->offset;
> + }
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> + rte_errno = EPERM;
> + return -1;
> + }
> +
> + if (req == SIZE_MAX) {
> + for (offset = 0;
> + offset < sizeof(struct rte_mbuf);
> + offset++) {
> + if (check_offset(offset, params->size,
> + params->align) == 0 &&
> + shm->free_space[offset] <
> best_zone) {
> + best_zone = shm->free_space[offset];
> + req = offset;
> + }
> + }
> + if (req == SIZE_MAX) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> + } else {
> + if (check_offset(req, params->size, params->align) < 0) {
> + rte_errno = EBUSY;
> + return -1;
> + }
> + }
> +
> + offset = req;
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> +
> + te = rte_zmalloc("MBUF_DYNFIELD_TAILQ_ENTRY", sizeof(*te), 0);
> + if (te == NULL)
> + return -1;
> +
> + mbuf_dynfield = rte_zmalloc("mbuf_dynfield",
> sizeof(*mbuf_dynfield), 0);
> + if (mbuf_dynfield == NULL) {
> + rte_free(te);
> + return -1;
> + }
> +
> + ret = strlcpy(mbuf_dynfield->params.name, params->name,
> + sizeof(mbuf_dynfield->params.name));
> + if (ret < 0 || ret >= (int)sizeof(mbuf_dynfield->params.name)) {
> + rte_errno = ENAMETOOLONG;
> + rte_free(mbuf_dynfield);
> + rte_free(te);
> + return -1;
> + }
> + memcpy(&mbuf_dynfield->params, params, sizeof(mbuf_dynfield-
> >params));
> + mbuf_dynfield->offset = offset;
> + te->data = mbuf_dynfield;
> +
> + TAILQ_INSERT_TAIL(mbuf_dynfield_list, te, next);
> +
> + for (i = offset; i < offset + params->size; i++)
> + shm->free_space[i] = 0;
> + process_score();
> +
> + RTE_LOG(DEBUG, MBUF, "Registered dynamic field %s (sz=%zu,
> al=%zu, fl=0x%x) -> %zd\n",
> + params->name, params->size, params->align, params->flags,
> + offset);
> +
> + return offset;
> +}
> +
> +int
> +rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> *params,
> + size_t req)
> +{
> + int ret;
> +
> + if (params->size >= sizeof(struct rte_mbuf)) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (!rte_is_power_of_2(params->align)) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (params->flags != 0) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_write_lock();
> + ret = __rte_mbuf_dynfield_register_offset(params, req);
> + rte_mcfg_tailq_write_unlock();
> +
> + return ret;
> +}
> +
> +int
> +rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params) {
> + return rte_mbuf_dynfield_register_offset(params, SIZE_MAX); }
> +
> +/* assume tailq is locked */
> +static struct mbuf_dynflag_elt *
> +__mbuf_dynflag_lookup(const char *name) {
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *mbuf_dynflag;
> + struct rte_tailq_entry *te;
> +
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> +
> + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> + mbuf_dynflag = (struct mbuf_dynflag_elt *)te->data;
> + if (strncmp(name, mbuf_dynflag->params.name,
> + RTE_MBUF_DYN_NAMESIZE) == 0)
> + break;
> + }
> +
> + if (te == NULL) {
> + rte_errno = ENOENT;
> + return NULL;
> + }
> +
> + return mbuf_dynflag;
> +}
> +
> +int
> +rte_mbuf_dynflag_lookup(const char *name,
> + struct rte_mbuf_dynflag *params)
> +{
> + struct mbuf_dynflag_elt *mbuf_dynflag;
> +
> + if (shm == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_read_lock();
> + mbuf_dynflag = __mbuf_dynflag_lookup(name);
> + rte_mcfg_tailq_read_unlock();
> +
> + if (mbuf_dynflag == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + if (params != NULL)
> + memcpy(params, &mbuf_dynflag->params,
> sizeof(*params));
> +
> + return mbuf_dynflag->bitnum;
> +}
> +
> +static int mbuf_dynflag_cmp(const struct rte_mbuf_dynflag *params1,
> + const struct rte_mbuf_dynflag *params2) {
> + if (strcmp(params1->name, params2->name))
> + return -1;
> + if (params1->flags != params2->flags)
> + return -1;
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static int
> +__rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag
> *params,
> + unsigned int req)
> +{
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *mbuf_dynflag = NULL;
> + struct rte_tailq_entry *te = NULL;
> + unsigned int bitnum;
> + int ret;
> +
> + if (shm == NULL && init_shared_mem() < 0)
> + return -1;
> +
> + mbuf_dynflag = __mbuf_dynflag_lookup(params->name);
> + if (mbuf_dynflag != NULL) {
> + if (req != UINT_MAX && req != mbuf_dynflag->bitnum) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + if (mbuf_dynflag_cmp(params, &mbuf_dynflag->params) <
> 0) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + return mbuf_dynflag->bitnum;
> + }
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> + rte_errno = EPERM;
> + return -1;
> + }
> +
> + if (req == UINT_MAX) {
> + if (shm->free_flags == 0) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> + bitnum = rte_bsf64(shm->free_flags);
> + } else {
> + if ((shm->free_flags & (1ULL << req)) == 0) {
> + rte_errno = EBUSY;
> + return -1;
> + }
> + bitnum = req;
> + }
> +
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> +
> + te = rte_zmalloc("MBUF_DYNFLAG_TAILQ_ENTRY", sizeof(*te), 0);
> + if (te == NULL)
> + return -1;
> +
> + mbuf_dynflag = rte_zmalloc("mbuf_dynflag",
> sizeof(*mbuf_dynflag), 0);
> + if (mbuf_dynflag == NULL) {
> + rte_free(te);
> + return -1;
> + }
> +
> + ret = strlcpy(mbuf_dynflag->params.name, params->name,
> + sizeof(mbuf_dynflag->params.name));
> + if (ret < 0 || ret >= (int)sizeof(mbuf_dynflag->params.name)) {
> + rte_free(mbuf_dynflag);
> + rte_free(te);
> + rte_errno = ENAMETOOLONG;
> + return -1;
> + }
> + mbuf_dynflag->bitnum = bitnum;
> + te->data = mbuf_dynflag;
> +
> + TAILQ_INSERT_TAIL(mbuf_dynflag_list, te, next);
> +
> + shm->free_flags &= ~(1ULL << bitnum);
> +
> + RTE_LOG(DEBUG, MBUF, "Registered dynamic flag %s (fl=0x%x) ->
> %u\n",
> + params->name, params->flags, bitnum);
> +
> + return bitnum;
> +}
> +
> +int
> +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag
> *params,
> + unsigned int req)
> +{
> + int ret;
> +
> + if (req != UINT_MAX && req >= 64) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_write_lock();
> + ret = __rte_mbuf_dynflag_register_bitnum(params, req);
> + rte_mcfg_tailq_write_unlock();
> +
> + return ret;
> +}
> +
> +int
> +rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params) {
> + return rte_mbuf_dynflag_register_bitnum(params, UINT_MAX); }
> +
> +void rte_mbuf_dyn_dump(FILE *out)
> +{
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *dynfield;
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *dynflag;
> + struct rte_tailq_entry *te;
> + size_t i;
> +
> + rte_mcfg_tailq_write_lock();
> + init_shared_mem();
> + fprintf(out, "Reserved fields:\n");
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> + dynfield = (struct mbuf_dynfield_elt *)te->data;
> + fprintf(out, " name=%s offset=%zd size=%zd align=%zd
> flags=%x\n",
> + dynfield->params.name, dynfield->offset,
> + dynfield->params.size, dynfield->params.align,
> + dynfield->params.flags);
> + }
> + fprintf(out, "Reserved flags:\n");
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> + dynflag = (struct mbuf_dynflag_elt *)te->data;
> + fprintf(out, " name=%s bitnum=%u flags=%x\n",
> + dynflag->params.name, dynflag->bitnum,
> + dynflag->params.flags);
> + }
> + fprintf(out, "Free space in mbuf (0 = free, value = zone
> alignment):\n");
> + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> + if ((i % 8) == 0)
> + fprintf(out, " %4.4zx: ", i);
> + fprintf(out, "%2.2x%s", shm->free_space[i],
> + (i % 8 != 7) ? " " : "\n");
> + }
> + rte_mcfg_tailq_write_unlock();
> +}
> diff --git a/lib/librte_mbuf/rte_mbuf_dyn.h
> b/lib/librte_mbuf/rte_mbuf_dyn.h new file mode 100644 index
> 000000000..307613c96
> --- /dev/null
> +++ b/lib/librte_mbuf/rte_mbuf_dyn.h
> @@ -0,0 +1,226 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 6WIND S.A.
> + */
> +
> +#ifndef _RTE_MBUF_DYN_H_
> +#define _RTE_MBUF_DYN_H_
> +
> +/**
> + * @file
> + * RTE Mbuf dynamic fields and flags
> + *
> + * Many features require to store data inside the mbuf. As the room in
> + * mbuf structure is limited, it is not possible to have a field for
> + * each feature. Also, changing fields in the mbuf structure can break
> + * the API or ABI.
> + *
> + * This module addresses this issue, by enabling the dynamic
> + * registration of fields or flags:
> + *
> + * - a dynamic field is a named area in the rte_mbuf structure, with a
> + * given size (>= 1 byte) and alignment constraint.
> + * - a dynamic flag is a named bit in the rte_mbuf structure, stored
> + * in mbuf->ol_flags.
> + *
> + * The typical use case is when a specific offload feature requires to
> + * register a dedicated offload field in the mbuf structure, and adding
> + * a static field or flag is not justified.
> + *
> + * Example of use:
> + *
> + * - A rte_mbuf_dynfield structure is defined, containing the parameters
> + * of the dynamic field to be registered:
> + * const struct rte_mbuf_dynfield rte_dynfield_my_feature = { ... };
> + * - The application initializes the PMD, and asks for this feature
> + * at port initialization by passing DEV_RX_OFFLOAD_MY_FEATURE in
> + * rxconf. This will make the PMD to register the field by calling
> + * rte_mbuf_dynfield_register(&rte_dynfield_my_feature). The PMD
> + * stores the returned offset.
> + * - The application that uses the offload feature also registers
> + * the field to retrieve the same offset.
> + * - When the PMD receives a packet, it can set the field:
> + * *RTE_MBUF_DYNFIELD(m, offset, <type *>) = value;
> + * - In the main loop, the application can retrieve the value with
> + * the same macro.
> + *
> + * To avoid wasting space, the dynamic fields or flags must only be
> + * reserved on demand, when an application asks for the related feature.
> + *
> + * The registration can be done at any moment, but it is not possible
> + * to unregister fields or flags for now.
> + *
> + * A dynamic field can be reserved and used by an application only.
> + * It can for instance be a packet mark.
> + */
> +
> +#include <sys/types.h>
> +/**
> + * Maximum length of the dynamic field or flag string.
> + */
> +#define RTE_MBUF_DYN_NAMESIZE 64
> +
> +/**
> + * Structure describing the parameters of a mbuf dynamic field.
> + */
> +struct rte_mbuf_dynfield {
> + char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the field. */
> + size_t size; /**< The number of bytes to reserve. */
> + size_t align; /**< The alignment constraint (power of 2). */
> + unsigned int flags; /**< Reserved for future use, must be 0. */ };
> +
> +/**
> + * Structure describing the parameters of a mbuf dynamic flag.
> + */
> +struct rte_mbuf_dynflag {
> + char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the dynamic
> flag. */
> + unsigned int flags; /**< Reserved for future use, must be 0. */ };
> +
> +/**
> + * Register space for a dynamic field in the mbuf structure.
> + *
> + * If the field is already registered (same name and parameters), its
> + * offset is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters (name, size,
> + * alignment constraint and flags).
> + * @return
> + * The offset in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, or flags).
> + * - EEXIST: this name is already register with different parameters.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: not enough room in mbuf.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name does not ends with \0.
> + */
> +__rte_experimental
> +int rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params);
> +
> +/**
> + * Register space for a dynamic field in the mbuf structure at offset.
> + *
> + * If the field is already registered (same name, parameters and
> +offset),
> + * the offset is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters (name, size,
> + * alignment constraint and flags).
> + * @param offset
> + * The requested offset. Ignored if SIZE_MAX is passed.
> + * @return
> + * The offset in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, flags, or offset).
> + * - EEXIST: this name is already register with different parameters.
> + * - EBUSY: the requested offset cannot be used.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: not enough room in mbuf.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name does not ends with \0.
> + */
> +__rte_experimental
> +int rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> *params,
> + size_t offset);
> +
> +/**
> + * Lookup for a registered dynamic mbuf field.
> + *
> + * @param name
> + * A string identifying the dynamic field.
> + * @param params
> + * If not NULL, and if the lookup is successful, the structure is
> + * filled with the parameters of the dynamic field.
> + * @return
> + * The offset of this field in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - ENOENT: no dynamic field matches this name.
> + */
> +__rte_experimental
> +int rte_mbuf_dynfield_lookup(const char *name,
> + struct rte_mbuf_dynfield *params);
> +
> +/**
> + * Register a dynamic flag in the mbuf structure.
> + *
> + * If the flag is already registered (same name and parameters), its
> + * bitnum is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters of the dynamic
> + * flag (name and options).
> + * @return
> + * The number of the reserved bit, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, or flags).
> + * - EEXIST: this name is already register with different parameters.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: no more flag available.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE -
> 1.
> + */
> +__rte_experimental
> +int rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params);
> +
> +/**
> + * Register a dynamic flag in the mbuf structure specifying bitnum.
> + *
> + * If the flag is already registered (same name, parameters and
> +bitnum),
> + * the bitnum is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters of the dynamic
> + * flag (name and options).
> + * @param bitnum
> + * The requested bitnum. Ignored if UINT_MAX is passed.
> + * @return
> + * The number of the reserved bit, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, or flags).
> + * - EEXIST: this name is already register with different parameters.
> + * - EBUSY: the requested bitnum cannot be used.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: no more flag available.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE -
> 1.
> + */
> +__rte_experimental
> +int rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag
> *params,
> + unsigned int bitnum);
> +
> +/**
> + * Lookup for a registered dynamic mbuf flag.
> + *
> + * @param name
> + * A string identifying the dynamic flag.
> + * @param params
> + * If not NULL, and if the lookup is successful, the structure is
> + * filled with the parameters of the dynamic flag.
> + * @return
> + * The offset of this flag in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - ENOENT: no dynamic flag matches this name.
> + */
> +__rte_experimental
> +int rte_mbuf_dynflag_lookup(const char *name,
> + struct rte_mbuf_dynflag *params);
> +
> +/**
> + * Helper macro to access to a dynamic field.
> + */
> +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) +
> +(offset)))
> +
> +/**
> + * Dump the status of dynamic fields and flags.
> + *
> + * @param out
> + * The stream where the status is displayed.
> + */
> +__rte_experimental
> +void rte_mbuf_dyn_dump(FILE *out);
> +
> +/* Placeholder for dynamic fields and flags declarations. */
> +
> +#endif
> diff --git a/lib/librte_mbuf/rte_mbuf_version.map
> b/lib/librte_mbuf/rte_mbuf_version.map
> index 519fead35..9bf5ca37a 100644
> --- a/lib/librte_mbuf/rte_mbuf_version.map
> +++ b/lib/librte_mbuf/rte_mbuf_version.map
> @@ -58,6 +58,13 @@ EXPERIMENTAL {
> global:
>
> rte_mbuf_check;
> + rte_mbuf_dynfield_lookup;
> + rte_mbuf_dynfield_register;
> + rte_mbuf_dynfield_register_offset;
> + rte_mbuf_dynflag_lookup;
> + rte_mbuf_dynflag_register;
> + rte_mbuf_dynflag_register_bitnum;
> + rte_mbuf_dyn_dump;
> rte_pktmbuf_copy;
>
> } DPDK_18.08;
> --
> 2.20.1
Hi Shahaf,
On Wed, Oct 23, 2019 at 12:00:30PM +0000, Shahaf Shuler wrote:
> Hi Olivier,
>
> Thursday, October 17, 2019 5:42 PM, Olivier Matz:
> > Subject: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and flags
> >
> > Many features require to store data inside the mbuf. As the room in mbuf
> > structure is limited, it is not possible to have a field for each feature. Also,
> > changing fields in the mbuf structure can break the API or ABI.
> >
> > This commit addresses these issues, by enabling the dynamic registration of
> > fields or flags:
> >
> > - a dynamic field is a named area in the rte_mbuf structure, with a
> > given size (>= 1 byte) and alignment constraint.
> > - a dynamic flag is a named bit in the rte_mbuf structure.
> >
> > The typical use case is a PMD that registers space for an offload feature,
> > when the application requests to enable this feature. As the space in mbuf is
> > limited, the space should only be reserved if it is going to be used (i.e when
> > the application explicitly asks for it).
>
> According to description, the dynamic field enables custom application and supported PMDs to use the dynamic part of the mbuf for their specific needs.
> However the mechanism to report and activate the field/flag registration comes from the general OFFLOAD flags.
>
> Maybe it will be better to an option to query and select dynamic fields for PMD outside of the standard ethdev offload flags?
It is not mandatory to use the ethdev layer to register a dynamic field
or flag in the mbuf. It is just the typical use case.
It can also be enabled when using a library that have specific needs,
for instance, you call rte_reorder_init(), and it will register the
sequence number dynamic field.
An application that requires a specific mbuf field can also do the
registration by itself.
In other words, when you initialize a subpart that needs a dynamic field
or flag, you have to do the registration there.
>
> >
> > The registration can be done at any moment, but it is not possible to
> > unregister fields or flags for now.
> >
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
On Wed, 23 Oct 2019 12:21:43 +0200
Olivier Matz <olivier.matz@6wind.com> wrote:
> On Wed, Oct 23, 2019 at 03:16:13AM +0000, Wang, Haiyue wrote:
> > > -----Original Message-----
> > > From: Ananyev, Konstantin
> > > Sent: Wednesday, October 23, 2019 06:52
> > > To: Olivier Matz <olivier.matz@6wind.com>; dev@dpdk.org
> > > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>; Wang,
> > > Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > > <keith.wiles@intel.com>; Morten Brørup <mb@smartsharesystems.com>; Stephen Hemminger
> > > <stephen@networkplumber.org>; Thomas Monjalon <thomas@monjalon.net>
> > > Subject: RE: [PATCH v2] mbuf: support dynamic fields and flags
> > >
> > >
> > > > Many features require to store data inside the mbuf. As the room in mbuf
> > > > structure is limited, it is not possible to have a field for each
> > > > feature. Also, changing fields in the mbuf structure can break the API
> > > > or ABI.
> > > >
> > > > This commit addresses these issues, by enabling the dynamic registration
> > > > of fields or flags:
> > > >
> > > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > > given size (>= 1 byte) and alignment constraint.
> > > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > > >
> > > > The typical use case is a PMD that registers space for an offload
> > > > feature, when the application requests to enable this feature. As
> > > > the space in mbuf is limited, the space should only be reserved if it
> > > > is going to be used (i.e when the application explicitly asks for it).
> > > >
> > > > The registration can be done at any moment, but it is not possible
> > > > to unregister fields or flags for now.
> > > >
> > > > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > > > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > > > ---
> > > >
> > > > v2
> > > >
> > > > * Rebase on top of master: solve conflict with Stephen's patchset
> > > > (packet copy)
> > > > * Add new apis to register a dynamic field/flag at a specific place
> > > > * Add a dump function (sugg by David)
> > > > * Enhance field registration function to select the best offset, keeping
> > > > large aligned zones as much as possible (sugg by Konstantin)
> > > > * Use a size_t and unsigned int instead of int when relevant
> > > > (sugg by Konstantin)
> > > > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > > > (sugg by Konstantin)
> > > > * Remove unused argument in private function (sugg by Konstantin)
> > > > * Fix and simplify locking (sugg by Konstantin)
> > > > * Fix minor typo
> > > >
> > > > rfc -> v1
> > > >
> > > > * Rebase on top of master
> > > > * Change registration API to use a structure instead of
> > > > variables, getting rid of #defines (Stephen's comment)
> > > > * Update flag registration to use a similar API as fields.
> > > > * Change max name length from 32 to 64 (sugg. by Thomas)
> > > > * Enhance API documentation (Haiyue's and Andrew's comments)
> > > > * Add a debug log at registration
> > > > * Add some words in release note
> > > > * Did some performance tests (sugg. by Andrew):
> > > > On my platform, reading a dynamic field takes ~3 cycles more
> > > > than a static field, and ~2 cycles more for writing.
> > > >
> > > > app/test/test_mbuf.c | 145 ++++++-
> > > > doc/guides/rel_notes/release_19_11.rst | 7 +
> > > > lib/librte_mbuf/Makefile | 2 +
> > > > lib/librte_mbuf/meson.build | 6 +-
> > > > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > > > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > > > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > > > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > > > 8 files changed, 959 insertions(+), 5 deletions(-)
> > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> > > >
> > > > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > > > index b9c2b2500..01cafad59 100644
> > > > --- a/app/test/test_mbuf.c
> > > > +++ b/app/test/test_mbuf.c
> > > > @@ -28,6 +28,7 @@
> > > > #include <rte_random.h>
> > > > #include <rte_cycles.h>
> > > > #include <rte_malloc.h>
> > > > +#include <rte_mbuf_dyn.h>
> > > >
> >
> > [snip]
> > > > +int
> > > > +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > > > + unsigned int req)
> > > > +{
> > > > + int ret;
> > > > +
> > > > + if (req != UINT_MAX && req >= 64) {
> > >
> > > Might be better to replace 64 with something like sizeof(mbuf->ol_flags) * CHAR_BIT or so.
> >
> > Might introduce a new macro like kernel:
> >
> > /**
> > * FIELD_SIZEOF - get the size of a struct's field
> > * @t: the target struct
> > * @f: the target struct's field
> > * Return: the size of @f in the struct definition without having a
> > * declared instance of @t.
> > */
> > #define FIELD_SIZEOF(t, f) (sizeof(((t*)0)->f))
> >
> > Then: FIELD_SIZEOF(rte_mbuf, ol_flags) * CHAR_BIT
>
> Good idea, thanks
>
Kernel is replacing FIELD_SIZEOF with sizeof_member
> -----Original Message-----
> From: Stephen Hemminger [mailto:stephen@networkplumber.org]
> Sent: Wednesday, October 23, 2019 23:00
> To: Olivier Matz <olivier.matz@6wind.com>
> Cc: Wang, Haiyue <haiyue.wang@intel.com>; Ananyev, Konstantin <konstantin.ananyev@intel.com>;
> dev@dpdk.org; Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce
> <bruce.richardson@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> <keith.wiles@intel.com>; Morten Brørup <mb@smartsharesystems.com>; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: Re: [PATCH v2] mbuf: support dynamic fields and flags
>
> On Wed, 23 Oct 2019 12:21:43 +0200
> Olivier Matz <olivier.matz@6wind.com> wrote:
>
> > On Wed, Oct 23, 2019 at 03:16:13AM +0000, Wang, Haiyue wrote:
> > > > -----Original Message-----
> > > > From: Ananyev, Konstantin
> > > > Sent: Wednesday, October 23, 2019 06:52
> > > > To: Olivier Matz <olivier.matz@6wind.com>; dev@dpdk.org
> > > > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Richardson, Bruce <bruce.richardson@intel.com>;
> Wang,
> > > > Haiyue <haiyue.wang@intel.com>; Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > > > <keith.wiles@intel.com>; Morten Brørup <mb@smartsharesystems.com>; Stephen Hemminger
> > > > <stephen@networkplumber.org>; Thomas Monjalon <thomas@monjalon.net>
> > > > Subject: RE: [PATCH v2] mbuf: support dynamic fields and flags
> > > >
> > > >
> > > > > Many features require to store data inside the mbuf. As the room in mbuf
> > > > > structure is limited, it is not possible to have a field for each
> > > > > feature. Also, changing fields in the mbuf structure can break the API
> > > > > or ABI.
> > > > >
> > > > > This commit addresses these issues, by enabling the dynamic registration
> > > > > of fields or flags:
> > > > >
> > > > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > > > given size (>= 1 byte) and alignment constraint.
> > > > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > > > >
> > > > > The typical use case is a PMD that registers space for an offload
> > > > > feature, when the application requests to enable this feature. As
> > > > > the space in mbuf is limited, the space should only be reserved if it
> > > > > is going to be used (i.e when the application explicitly asks for it).
> > > > >
> > > > > The registration can be done at any moment, but it is not possible
> > > > > to unregister fields or flags for now.
> > > > >
> > > > > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > > > > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > > > > ---
> > > > >
> > > > > v2
> > > > >
> > > > > * Rebase on top of master: solve conflict with Stephen's patchset
> > > > > (packet copy)
> > > > > * Add new apis to register a dynamic field/flag at a specific place
> > > > > * Add a dump function (sugg by David)
> > > > > * Enhance field registration function to select the best offset, keeping
> > > > > large aligned zones as much as possible (sugg by Konstantin)
> > > > > * Use a size_t and unsigned int instead of int when relevant
> > > > > (sugg by Konstantin)
> > > > > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > > > > (sugg by Konstantin)
> > > > > * Remove unused argument in private function (sugg by Konstantin)
> > > > > * Fix and simplify locking (sugg by Konstantin)
> > > > > * Fix minor typo
> > > > >
> > > > > rfc -> v1
> > > > >
> > > > > * Rebase on top of master
> > > > > * Change registration API to use a structure instead of
> > > > > variables, getting rid of #defines (Stephen's comment)
> > > > > * Update flag registration to use a similar API as fields.
> > > > > * Change max name length from 32 to 64 (sugg. by Thomas)
> > > > > * Enhance API documentation (Haiyue's and Andrew's comments)
> > > > > * Add a debug log at registration
> > > > > * Add some words in release note
> > > > > * Did some performance tests (sugg. by Andrew):
> > > > > On my platform, reading a dynamic field takes ~3 cycles more
> > > > > than a static field, and ~2 cycles more for writing.
> > > > >
> > > > > app/test/test_mbuf.c | 145 ++++++-
> > > > > doc/guides/rel_notes/release_19_11.rst | 7 +
> > > > > lib/librte_mbuf/Makefile | 2 +
> > > > > lib/librte_mbuf/meson.build | 6 +-
> > > > > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > > > > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > > > > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > > > > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > > > > 8 files changed, 959 insertions(+), 5 deletions(-)
> > > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.c
> > > > > create mode 100644 lib/librte_mbuf/rte_mbuf_dyn.h
> > > > >
> > > > > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c
> > > > > index b9c2b2500..01cafad59 100644
> > > > > --- a/app/test/test_mbuf.c
> > > > > +++ b/app/test/test_mbuf.c
> > > > > @@ -28,6 +28,7 @@
> > > > > #include <rte_random.h>
> > > > > #include <rte_cycles.h>
> > > > > #include <rte_malloc.h>
> > > > > +#include <rte_mbuf_dyn.h>
> > > > >
> > >
> > > [snip]
> > > > > +int
> > > > > +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > > > > + unsigned int req)
> > > > > +{
> > > > > + int ret;
> > > > > +
> > > > > + if (req != UINT_MAX && req >= 64) {
> > > >
> > > > Might be better to replace 64 with something like sizeof(mbuf->ol_flags) * CHAR_BIT or so.
> > >
> > > Might introduce a new macro like kernel:
> > >
> > > /**
> > > * FIELD_SIZEOF - get the size of a struct's field
> > > * @t: the target struct
> > > * @f: the target struct's field
> > > * Return: the size of @f in the struct definition without having a
> > > * declared instance of @t.
> > > */
> > > #define FIELD_SIZEOF(t, f) (sizeof(((t*)0)->f))
> > >
> > > Then: FIELD_SIZEOF(rte_mbuf, ol_flags) * CHAR_BIT
> >
> > Good idea, thanks
> >
>
> Kernel is replacing FIELD_SIZEOF with sizeof_member
Yes, but looks like in 5.5 ? 5.4 hasn't merged. ;-)
https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.4-Size-Of-Member
https://patchwork.kernel.org/patch/11184583/
+/**
+ * sizeof_member(TYPE, MEMBER) - get the size of a struct's member
+ *
+ * @TYPE: the target struct
+ * @MEMBER: the target struct's member
+ *
+ * Return: the size of @MEMBER in the struct definition without having a
+ * declared instance of @TYPE.
+ */
+#define sizeof_member(TYPE, MEMBER) (sizeof(((TYPE *)0)->MEMBER))
Wednesday, October 23, 2019 4:34 PM, Olivier Matz:
> Subject: Re: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and flags
>
> Hi Shahaf,
>
> On Wed, Oct 23, 2019 at 12:00:30PM +0000, Shahaf Shuler wrote:
> > Hi Olivier,
> >
> > Thursday, October 17, 2019 5:42 PM, Olivier Matz:
> > > Subject: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and
> > > flags
> > >
> > > Many features require to store data inside the mbuf. As the room in
> > > mbuf structure is limited, it is not possible to have a field for
> > > each feature. Also, changing fields in the mbuf structure can break the
> API or ABI.
> > >
> > > This commit addresses these issues, by enabling the dynamic
> > > registration of fields or flags:
> > >
> > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > given size (>= 1 byte) and alignment constraint.
> > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > >
> > > The typical use case is a PMD that registers space for an offload
> > > feature, when the application requests to enable this feature. As
> > > the space in mbuf is limited, the space should only be reserved if
> > > it is going to be used (i.e when the application explicitly asks for it).
> >
> > According to description, the dynamic field enables custom application and
> supported PMDs to use the dynamic part of the mbuf for their specific
> needs.
> > However the mechanism to report and activate the field/flag registration
> comes from the general OFFLOAD flags.
> >
> > Maybe it will be better to an option to query and select dynamic fields for
> PMD outside of the standard ethdev offload flags?
>
> It is not mandatory to use the ethdev layer to register a dynamic field or flag
> in the mbuf. It is just the typical use case.
>
> It can also be enabled when using a library that have specific needs, for
> instance, you call rte_reorder_init(), and it will register the sequence number
> dynamic field.
>
> An application that requires a specific mbuf field can also do the registration
> by itself.
>
> In other words, when you initialize a subpart that needs a dynamic field or
> flag, you have to do the registration there.
>
I guess my question mainly targets one of the use cases for dynamic mbuf fields which is vendor specific offloads.
On such case we would like to have dynamic fields/flags negotiated between the application and PMD.
The question is whether we provide a unified way for application to query PMD specific dynamic fields or we let PMD vendor to implement this handshake as they wish (devargs, through PMD doc, etc..)
Hi,
On Thu, Oct 24, 2019 at 04:54:20AM +0000, Shahaf Shuler wrote:
> Wednesday, October 23, 2019 4:34 PM, Olivier Matz:
> > Subject: Re: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and flags
> >
> > Hi Shahaf,
> >
> > On Wed, Oct 23, 2019 at 12:00:30PM +0000, Shahaf Shuler wrote:
> > > Hi Olivier,
> > >
> > > Thursday, October 17, 2019 5:42 PM, Olivier Matz:
> > > > Subject: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and
> > > > flags
> > > >
> > > > Many features require to store data inside the mbuf. As the room in
> > > > mbuf structure is limited, it is not possible to have a field for
> > > > each feature. Also, changing fields in the mbuf structure can break the
> > API or ABI.
> > > >
> > > > This commit addresses these issues, by enabling the dynamic
> > > > registration of fields or flags:
> > > >
> > > > - a dynamic field is a named area in the rte_mbuf structure, with a
> > > > given size (>= 1 byte) and alignment constraint.
> > > > - a dynamic flag is a named bit in the rte_mbuf structure.
> > > >
> > > > The typical use case is a PMD that registers space for an offload
> > > > feature, when the application requests to enable this feature. As
> > > > the space in mbuf is limited, the space should only be reserved if
> > > > it is going to be used (i.e when the application explicitly asks for it).
> > >
> > > According to description, the dynamic field enables custom application and
> > supported PMDs to use the dynamic part of the mbuf for their specific
> > needs.
> > > However the mechanism to report and activate the field/flag registration
> > comes from the general OFFLOAD flags.
> > >
> > > Maybe it will be better to an option to query and select dynamic fields for
> > PMD outside of the standard ethdev offload flags?
> >
> > It is not mandatory to use the ethdev layer to register a dynamic field or flag
> > in the mbuf. It is just the typical use case.
> >
> > It can also be enabled when using a library that have specific needs, for
> > instance, you call rte_reorder_init(), and it will register the sequence number
> > dynamic field.
> >
> > An application that requires a specific mbuf field can also do the registration
> > by itself.
> >
> > In other words, when you initialize a subpart that needs a dynamic field or
> > flag, you have to do the registration there.
> >
>
> I guess my question mainly targets one of the use cases for dynamic mbuf fields which is vendor specific offloads.
> On such case we would like to have dynamic fields/flags negotiated between the application and PMD.
>
> The question is whether we provide a unified way for application to query PMD specific dynamic fields or we let PMD vendor to implement this handshake as they wish (devargs, through PMD doc, etc..)
I have no strong opinion. It can be a PMD-specific API (function or
devargs) to enable the feature.
The only important thing is to not register the field if it won't be
used.
Hi,
Doc building failed, it seems the rte_mbuf_dynfield_copy() description should be fixed:
./lib/librte_mbuf/rte_mbuf.h:1694: warning: argument 'm_dst' of command @param is not found in the argument list of rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
./lib/librte_mbuf/rte_mbuf.h:1694: warning: argument 'm_src' of command @param is not found in the argument list of rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
./lib/librte_mbuf/rte_mbuf.h:1694: warning: The following parameters of rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc) are not documented
With best regards,
Slava
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Olivier Matz
> Sent: Thursday, October 17, 2019 17:42
> To: dev@dpdk.org
> Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Bruce Richardson
> <bruce.richardson@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
> Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> <keith.wiles@intel.com>; Ananyev, Konstantin
> <konstantin.ananyev@intel.com>; Morten Brørup
> <mb@smartsharesystems.com>; Stephen Hemminger
> <stephen@networkplumber.org>; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and flags
>
> Many features require to store data inside the mbuf. As the room in mbuf
> structure is limited, it is not possible to have a field for each feature. Also,
> changing fields in the mbuf structure can break the API or ABI.
>
> This commit addresses these issues, by enabling the dynamic registration of
> fields or flags:
>
> - a dynamic field is a named area in the rte_mbuf structure, with a
> given size (>= 1 byte) and alignment constraint.
> - a dynamic flag is a named bit in the rte_mbuf structure.
>
> The typical use case is a PMD that registers space for an offload feature,
> when the application requests to enable this feature. As the space in mbuf is
> limited, the space should only be reserved if it is going to be used (i.e when
> the application explicitly asks for it).
>
> The registration can be done at any moment, but it is not possible to
> unregister fields or flags for now.
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> Acked-by: Thomas Monjalon <thomas@monjalon.net>
> ---
>
> v2
>
> * Rebase on top of master: solve conflict with Stephen's patchset
> (packet copy)
> * Add new apis to register a dynamic field/flag at a specific place
> * Add a dump function (sugg by David)
> * Enhance field registration function to select the best offset, keeping
> large aligned zones as much as possible (sugg by Konstantin)
> * Use a size_t and unsigned int instead of int when relevant
> (sugg by Konstantin)
> * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> (sugg by Konstantin)
> * Remove unused argument in private function (sugg by Konstantin)
> * Fix and simplify locking (sugg by Konstantin)
> * Fix minor typo
>
> rfc -> v1
>
> * Rebase on top of master
> * Change registration API to use a structure instead of
> variables, getting rid of #defines (Stephen's comment)
> * Update flag registration to use a similar API as fields.
> * Change max name length from 32 to 64 (sugg. by Thomas)
> * Enhance API documentation (Haiyue's and Andrew's comments)
> * Add a debug log at registration
> * Add some words in release note
> * Did some performance tests (sugg. by Andrew):
> On my platform, reading a dynamic field takes ~3 cycles more
> than a static field, and ~2 cycles more for writing.
>
> app/test/test_mbuf.c | 145 ++++++-
> doc/guides/rel_notes/release_19_11.rst | 7 +
> lib/librte_mbuf/Makefile | 2 +
> lib/librte_mbuf/meson.build | 6 +-
> lib/librte_mbuf/rte_mbuf.h | 23 +-
> lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> lib/librte_mbuf/rte_mbuf_version.map | 7 +
> 8 files changed, 959 insertions(+), 5 deletions(-) create mode 100644
> lib/librte_mbuf/rte_mbuf_dyn.c create mode 100644
> lib/librte_mbuf/rte_mbuf_dyn.h
>
> diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c index
> b9c2b2500..01cafad59 100644
> --- a/app/test/test_mbuf.c
> +++ b/app/test/test_mbuf.c
> @@ -28,6 +28,7 @@
> #include <rte_random.h>
> #include <rte_cycles.h>
> #include <rte_malloc.h>
> +#include <rte_mbuf_dyn.h>
>
> #include "test.h"
>
> @@ -657,7 +658,6 @@ test_attach_from_different_pool(struct
> rte_mempool *pktmbuf_pool,
> rte_pktmbuf_free(clone2);
> return -1;
> }
> -#undef GOTO_FAIL
>
> /*
> * test allocation and free of mbufs
> @@ -1276,6 +1276,143 @@ test_tx_offload(void)
> return (v1 == v2) ? 0 : -EINVAL;
> }
>
> +static int
> +test_mbuf_dyn(struct rte_mempool *pktmbuf_pool) {
> + const struct rte_mbuf_dynfield dynfield = {
> + .name = "test-dynfield",
> + .size = sizeof(uint8_t),
> + .align = __alignof__(uint8_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield2 = {
> + .name = "test-dynfield2",
> + .size = sizeof(uint16_t),
> + .align = __alignof__(uint16_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield3 = {
> + .name = "test-dynfield3",
> + .size = sizeof(uint8_t),
> + .align = __alignof__(uint8_t),
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield_fail_big = {
> + .name = "test-dynfield-fail-big",
> + .size = 256,
> + .align = 1,
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynfield dynfield_fail_align = {
> + .name = "test-dynfield-fail-align",
> + .size = 1,
> + .align = 3,
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag = {
> + .name = "test-dynflag",
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag2 = {
> + .name = "test-dynflag2",
> + .flags = 0,
> + };
> + const struct rte_mbuf_dynflag dynflag3 = {
> + .name = "test-dynflag3",
> + .flags = 0,
> + };
> + struct rte_mbuf *m = NULL;
> + int offset, offset2, offset3;
> + int flag, flag2, flag3;
> + int ret;
> +
> + printf("Test mbuf dynamic fields and flags\n");
> + rte_mbuf_dyn_dump(stdout);
> +
> + offset = rte_mbuf_dynfield_register(&dynfield);
> + if (offset == -1)
> + GOTO_FAIL("failed to register dynamic field, offset=%d: %s",
> + offset, strerror(errno));
> +
> + ret = rte_mbuf_dynfield_register(&dynfield);
> + if (ret != offset)
> + GOTO_FAIL("failed to lookup dynamic field, ret=%d: %s",
> + ret, strerror(errno));
> +
> + offset2 = rte_mbuf_dynfield_register(&dynfield2);
> + if (offset2 == -1 || offset2 == offset || (offset2 & 1))
> + GOTO_FAIL("failed to register dynamic field 2, offset2=%d:
> %s",
> + offset2, strerror(errno));
> +
> + offset3 = rte_mbuf_dynfield_register_offset(&dynfield3,
> + offsetof(struct rte_mbuf, dynfield1[1]));
> + if (offset3 != offsetof(struct rte_mbuf, dynfield1[1]))
> + GOTO_FAIL("failed to register dynamic field 3, offset=%d:
> %s",
> + offset3, strerror(errno));
> +
> + printf("dynfield: offset=%d, offset2=%d, offset3=%d\n",
> + offset, offset2, offset3);
> +
> + ret = rte_mbuf_dynfield_register(&dynfield_fail_big);
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (too big)");
> +
> + ret = rte_mbuf_dynfield_register(&dynfield_fail_align);
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (bad
> alignment)");
> +
> + ret = rte_mbuf_dynfield_register_offset(&dynfield_fail_align,
> + offsetof(struct rte_mbuf, ol_flags));
> + if (ret != -1)
> + GOTO_FAIL("dynamic field creation should fail (not avail)");
> +
> + flag = rte_mbuf_dynflag_register(&dynflag);
> + if (flag == -1)
> + GOTO_FAIL("failed to register dynamic flag, flag=%d: %s",
> + flag, strerror(errno));
> +
> + ret = rte_mbuf_dynflag_register(&dynflag);
> + if (ret != flag)
> + GOTO_FAIL("failed to lookup dynamic flag, ret=%d: %s",
> + ret, strerror(errno));
> +
> + flag2 = rte_mbuf_dynflag_register(&dynflag2);
> + if (flag2 == -1 || flag2 == flag)
> + GOTO_FAIL("failed to register dynamic flag 2, flag2=%d: %s",
> + flag2, strerror(errno));
> +
> + flag3 = rte_mbuf_dynflag_register_bitnum(&dynflag3,
> + rte_bsf64(PKT_LAST_FREE));
> + if (flag3 != rte_bsf64(PKT_LAST_FREE))
> + GOTO_FAIL("failed to register dynamic flag 3, flag2=%d: %s",
> + flag3, strerror(errno));
> +
> + printf("dynflag: flag=%d, flag2=%d, flag3=%d\n", flag, flag2, flag3);
> +
> + /* set, get dynamic field */
> + m = rte_pktmbuf_alloc(pktmbuf_pool);
> + if (m == NULL)
> + GOTO_FAIL("Cannot allocate mbuf");
> +
> + *RTE_MBUF_DYNFIELD(m, offset, uint8_t *) = 1;
> + if (*RTE_MBUF_DYNFIELD(m, offset, uint8_t *) != 1)
> + GOTO_FAIL("failed to read dynamic field");
> + *RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) = 1000;
> + if (*RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) != 1000)
> + GOTO_FAIL("failed to read dynamic field");
> +
> + /* set a dynamic flag */
> + m->ol_flags |= (1ULL << flag);
> +
> + rte_mbuf_dyn_dump(stdout);
> + rte_pktmbuf_free(m);
> + return 0;
> +fail:
> + rte_pktmbuf_free(m);
> + return -1;
> +}
> +#undef GOTO_FAIL
> +
> static int
> test_mbuf(void)
> {
> @@ -1295,6 +1432,12 @@ test_mbuf(void)
> goto err;
> }
>
> + /* test registration of dynamic fields and flags */
> + if (test_mbuf_dyn(pktmbuf_pool) < 0) {
> + printf("mbuf dynflag test failed\n");
> + goto err;
> + }
> +
> /* create a specific pktmbuf pool with a priv_size != 0 and no data
> * room size */
> pktmbuf_pool2 = rte_pktmbuf_pool_create("test_pktmbuf_pool2",
> diff --git a/doc/guides/rel_notes/release_19_11.rst
> b/doc/guides/rel_notes/release_19_11.rst
> index 85953b962..9e9c94554 100644
> --- a/doc/guides/rel_notes/release_19_11.rst
> +++ b/doc/guides/rel_notes/release_19_11.rst
> @@ -21,6 +21,13 @@ DPDK Release 19.11
>
> xdg-open build/doc/html/guides/rel_notes/release_19_11.html
>
> +* **Add support of support dynamic fields and flags in mbuf.**
> +
> + This new feature adds the ability to dynamically register some room
> + for a field or a flag in the mbuf structure. This is typically used
> + for specific offload features, where adding a static field or flag in
> + the mbuf is not justified.
> +
>
> New Features
> ------------
> diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile index
> c8f6d2689..5a9bcee73 100644
> --- a/lib/librte_mbuf/Makefile
> +++ b/lib/librte_mbuf/Makefile
> @@ -17,8 +17,10 @@ LIBABIVER := 5
>
> # all source are stored in SRCS-y
> SRCS-$(CONFIG_RTE_LIBRTE_MBUF) := rte_mbuf.c rte_mbuf_ptype.c
> rte_mbuf_pool_ops.c
> +SRCS-$(CONFIG_RTE_LIBRTE_MBUF) += rte_mbuf_dyn.c
>
> # install includes
> SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include := rte_mbuf.h
> rte_mbuf_ptype.h rte_mbuf_pool_ops.h
> +SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include += rte_mbuf_dyn.h
>
> include $(RTE_SDK)/mk/rte.lib.mk
> diff --git a/lib/librte_mbuf/meson.build b/lib/librte_mbuf/meson.build index
> 6cc11ebb4..9137e8f26 100644
> --- a/lib/librte_mbuf/meson.build
> +++ b/lib/librte_mbuf/meson.build
> @@ -2,8 +2,10 @@
> # Copyright(c) 2017 Intel Corporation
>
> version = 5
> -sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c') -
> headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h')
> +sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c',
> + 'rte_mbuf_dyn.c')
> +headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h',
> + 'rte_mbuf_dyn.h')
> deps += ['mempool']
>
> allow_experimental_apis = true
> diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index
> fb0849ac1..5740b1e93 100644
> --- a/lib/librte_mbuf/rte_mbuf.h
> +++ b/lib/librte_mbuf/rte_mbuf.h
> @@ -198,9 +198,12 @@ extern "C" {
> #define PKT_RX_OUTER_L4_CKSUM_GOOD (1ULL << 22)
> #define PKT_RX_OUTER_L4_CKSUM_INVALID ((1ULL << 21) | (1ULL << 22))
>
> -/* add new RX flags here */
> +/* add new RX flags here, don't forget to update PKT_FIRST_FREE */
>
> -/* add new TX flags here */
> +#define PKT_FIRST_FREE (1ULL << 23)
> +#define PKT_LAST_FREE (1ULL << 39)
> +
> +/* add new TX flags here, don't forget to update PKT_LAST_FREE */
>
> /**
> * Indicate that the metadata field in the mbuf is in use.
> @@ -738,6 +741,7 @@ struct rte_mbuf {
> */
> struct rte_mbuf_ext_shared_info *shinfo;
>
> + uint64_t dynfield1[2]; /**< Reserved for dynamic fields. */
> } __rte_cache_aligned;
>
> /**
> @@ -1684,6 +1688,20 @@ rte_pktmbuf_attach_extbuf(struct rte_mbuf *m,
> void *buf_addr,
> */
> #define rte_pktmbuf_detach_extbuf(m) rte_pktmbuf_detach(m)
>
> +/**
> + * Copy dynamic fields from m_src to m_dst.
> + *
> + * @param m_dst
> + * The destination mbuf.
> + * @param m_src
> + * The source mbuf.
> + */
> +static inline void
> +rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf
> +*msrc) {
> + memcpy(&mdst->dynfield1, msrc->dynfield1, sizeof(mdst-
> >dynfield1)); }
> +
> /* internal */
> static inline void
> __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf
> *msrc) @@ -1695,6 +1713,7 @@ __rte_pktmbuf_copy_hdr(struct rte_mbuf
> *mdst, const struct rte_mbuf *msrc)
> mdst->hash = msrc->hash;
> mdst->packet_type = msrc->packet_type;
> mdst->timestamp = msrc->timestamp;
> + rte_mbuf_dynfield_copy(mdst, msrc);
> }
>
> /**
> diff --git a/lib/librte_mbuf/rte_mbuf_dyn.c
> b/lib/librte_mbuf/rte_mbuf_dyn.c new file mode 100644 index
> 000000000..9ef235483
> --- /dev/null
> +++ b/lib/librte_mbuf/rte_mbuf_dyn.c
> @@ -0,0 +1,548 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 6WIND S.A.
> + */
> +
> +#include <sys/queue.h>
> +#include <stdint.h>
> +#include <limits.h>
> +
> +#include <rte_common.h>
> +#include <rte_eal.h>
> +#include <rte_eal_memconfig.h>
> +#include <rte_tailq.h>
> +#include <rte_errno.h>
> +#include <rte_malloc.h>
> +#include <rte_string_fns.h>
> +#include <rte_mbuf.h>
> +#include <rte_mbuf_dyn.h>
> +
> +#define RTE_MBUF_DYN_MZNAME "rte_mbuf_dyn"
> +
> +struct mbuf_dynfield_elt {
> + TAILQ_ENTRY(mbuf_dynfield_elt) next;
> + struct rte_mbuf_dynfield params;
> + size_t offset;
> +};
> +TAILQ_HEAD(mbuf_dynfield_list, rte_tailq_entry);
> +
> +static struct rte_tailq_elem mbuf_dynfield_tailq = {
> + .name = "RTE_MBUF_DYNFIELD",
> +};
> +EAL_REGISTER_TAILQ(mbuf_dynfield_tailq);
> +
> +struct mbuf_dynflag_elt {
> + TAILQ_ENTRY(mbuf_dynflag_elt) next;
> + struct rte_mbuf_dynflag params;
> + unsigned int bitnum;
> +};
> +TAILQ_HEAD(mbuf_dynflag_list, rte_tailq_entry);
> +
> +static struct rte_tailq_elem mbuf_dynflag_tailq = {
> + .name = "RTE_MBUF_DYNFLAG",
> +};
> +EAL_REGISTER_TAILQ(mbuf_dynflag_tailq);
> +
> +struct mbuf_dyn_shm {
> + /**
> + * For each mbuf byte, free_space[i] != 0 if space is free.
> + * The value is the size of the biggest aligned element that
> + * can fit in the zone.
> + */
> + uint8_t free_space[sizeof(struct rte_mbuf)];
> + /** Bitfield of available flags. */
> + uint64_t free_flags;
> +};
> +static struct mbuf_dyn_shm *shm;
> +
> +/* Set the value of free_space[] according to the size and alignment of
> + * the free areas. This helps to select the best place when reserving a
> + * dynamic field. Assume tailq is locked.
> + */
> +static void
> +process_score(void)
> +{
> + size_t off, align, size, i;
> +
> + /* first, erase previous info */
> + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> + if (shm->free_space[i])
> + shm->free_space[i] = 1;
> + }
> +
> + for (off = 0; off < sizeof(struct rte_mbuf); off++) {
> + /* get the size of the free zone */
> + for (size = 0; shm->free_space[off + size]; size++)
> + ;
> + if (size == 0)
> + continue;
> +
> + /* get the alignment of biggest object that can fit in
> + * the zone at this offset.
> + */
> + for (align = 1;
> + (off % (align << 1)) == 0 && (align << 1) <= size;
> + align <<= 1)
> + ;
> +
> + /* save it in free_space[] */
> + for (i = off; i < off + size; i++)
> + shm->free_space[i] = RTE_MAX(align, shm-
> >free_space[i]);
> + }
> +}
> +
> +/* Allocate and initialize the shared memory. Assume tailq is locked */
> +static int
> +init_shared_mem(void)
> +{
> + const struct rte_memzone *mz;
> + uint64_t mask;
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + mz =
> rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
> + sizeof(struct
> mbuf_dyn_shm),
> + SOCKET_ID_ANY, 0,
> + RTE_CACHE_LINE_SIZE);
> + } else {
> + mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
> + }
> + if (mz == NULL)
> + return -1;
> +
> + shm = mz->addr;
> +
> +#define mark_free(field) \
> + memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
> + 1, sizeof(((struct rte_mbuf *)0)->field))
> +
> + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> + /* init free_space, keep it sync'd with
> + * rte_mbuf_dynfield_copy().
> + */
> + memset(shm, 0, sizeof(*shm));
> + mark_free(dynfield1);
> +
> + /* init free_flags */
> + for (mask = PKT_FIRST_FREE; mask <= PKT_LAST_FREE; mask
> <<= 1)
> + shm->free_flags |= mask;
> +
> + process_score();
> + }
> +#undef mark_free
> +
> + return 0;
> +}
> +
> +/* check if this offset can be used */
> +static int
> +check_offset(size_t offset, size_t size, size_t align) {
> + size_t i;
> +
> + if ((offset & (align - 1)) != 0)
> + return -1;
> + if (offset + size > sizeof(struct rte_mbuf))
> + return -1;
> +
> + for (i = 0; i < size; i++) {
> + if (!shm->free_space[i + offset])
> + return -1;
> + }
> +
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static struct mbuf_dynfield_elt *
> +__mbuf_dynfield_lookup(const char *name) {
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *mbuf_dynfield;
> + struct rte_tailq_entry *te;
> +
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> +
> + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> + mbuf_dynfield = (struct mbuf_dynfield_elt *)te->data;
> + if (strcmp(name, mbuf_dynfield->params.name) == 0)
> + break;
> + }
> +
> + if (te == NULL) {
> + rte_errno = ENOENT;
> + return NULL;
> + }
> +
> + return mbuf_dynfield;
> +}
> +
> +int
> +rte_mbuf_dynfield_lookup(const char *name, struct rte_mbuf_dynfield
> +*params) {
> + struct mbuf_dynfield_elt *mbuf_dynfield;
> +
> + if (shm == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_read_lock();
> + mbuf_dynfield = __mbuf_dynfield_lookup(name);
> + rte_mcfg_tailq_read_unlock();
> +
> + if (mbuf_dynfield == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + if (params != NULL)
> + memcpy(params, &mbuf_dynfield->params,
> sizeof(*params));
> +
> + return mbuf_dynfield->offset;
> +}
> +
> +static int mbuf_dynfield_cmp(const struct rte_mbuf_dynfield *params1,
> + const struct rte_mbuf_dynfield *params2) {
> + if (strcmp(params1->name, params2->name))
> + return -1;
> + if (params1->size != params2->size)
> + return -1;
> + if (params1->align != params2->align)
> + return -1;
> + if (params1->flags != params2->flags)
> + return -1;
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static int
> +__rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> *params,
> + size_t req)
> +{
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *mbuf_dynfield = NULL;
> + struct rte_tailq_entry *te = NULL;
> + unsigned int best_zone = UINT_MAX;
> + size_t i, offset;
> + int ret;
> +
> + if (shm == NULL && init_shared_mem() < 0)
> + return -1;
> +
> + mbuf_dynfield = __mbuf_dynfield_lookup(params->name);
> + if (mbuf_dynfield != NULL) {
> + if (req != SIZE_MAX && req != mbuf_dynfield->offset) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + if (mbuf_dynfield_cmp(params, &mbuf_dynfield->params) <
> 0) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + return mbuf_dynfield->offset;
> + }
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> + rte_errno = EPERM;
> + return -1;
> + }
> +
> + if (req == SIZE_MAX) {
> + for (offset = 0;
> + offset < sizeof(struct rte_mbuf);
> + offset++) {
> + if (check_offset(offset, params->size,
> + params->align) == 0 &&
> + shm->free_space[offset] <
> best_zone) {
> + best_zone = shm->free_space[offset];
> + req = offset;
> + }
> + }
> + if (req == SIZE_MAX) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> + } else {
> + if (check_offset(req, params->size, params->align) < 0) {
> + rte_errno = EBUSY;
> + return -1;
> + }
> + }
> +
> + offset = req;
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> +
> + te = rte_zmalloc("MBUF_DYNFIELD_TAILQ_ENTRY", sizeof(*te), 0);
> + if (te == NULL)
> + return -1;
> +
> + mbuf_dynfield = rte_zmalloc("mbuf_dynfield",
> sizeof(*mbuf_dynfield), 0);
> + if (mbuf_dynfield == NULL) {
> + rte_free(te);
> + return -1;
> + }
> +
> + ret = strlcpy(mbuf_dynfield->params.name, params->name,
> + sizeof(mbuf_dynfield->params.name));
> + if (ret < 0 || ret >= (int)sizeof(mbuf_dynfield->params.name)) {
> + rte_errno = ENAMETOOLONG;
> + rte_free(mbuf_dynfield);
> + rte_free(te);
> + return -1;
> + }
> + memcpy(&mbuf_dynfield->params, params, sizeof(mbuf_dynfield-
> >params));
> + mbuf_dynfield->offset = offset;
> + te->data = mbuf_dynfield;
> +
> + TAILQ_INSERT_TAIL(mbuf_dynfield_list, te, next);
> +
> + for (i = offset; i < offset + params->size; i++)
> + shm->free_space[i] = 0;
> + process_score();
> +
> + RTE_LOG(DEBUG, MBUF, "Registered dynamic field %s (sz=%zu,
> al=%zu, fl=0x%x) -> %zd\n",
> + params->name, params->size, params->align, params->flags,
> + offset);
> +
> + return offset;
> +}
> +
> +int
> +rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> *params,
> + size_t req)
> +{
> + int ret;
> +
> + if (params->size >= sizeof(struct rte_mbuf)) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (!rte_is_power_of_2(params->align)) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> + if (params->flags != 0) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_write_lock();
> + ret = __rte_mbuf_dynfield_register_offset(params, req);
> + rte_mcfg_tailq_write_unlock();
> +
> + return ret;
> +}
> +
> +int
> +rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params) {
> + return rte_mbuf_dynfield_register_offset(params, SIZE_MAX); }
> +
> +/* assume tailq is locked */
> +static struct mbuf_dynflag_elt *
> +__mbuf_dynflag_lookup(const char *name) {
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *mbuf_dynflag;
> + struct rte_tailq_entry *te;
> +
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> +
> + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> + mbuf_dynflag = (struct mbuf_dynflag_elt *)te->data;
> + if (strncmp(name, mbuf_dynflag->params.name,
> + RTE_MBUF_DYN_NAMESIZE) == 0)
> + break;
> + }
> +
> + if (te == NULL) {
> + rte_errno = ENOENT;
> + return NULL;
> + }
> +
> + return mbuf_dynflag;
> +}
> +
> +int
> +rte_mbuf_dynflag_lookup(const char *name,
> + struct rte_mbuf_dynflag *params)
> +{
> + struct mbuf_dynflag_elt *mbuf_dynflag;
> +
> + if (shm == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_read_lock();
> + mbuf_dynflag = __mbuf_dynflag_lookup(name);
> + rte_mcfg_tailq_read_unlock();
> +
> + if (mbuf_dynflag == NULL) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> +
> + if (params != NULL)
> + memcpy(params, &mbuf_dynflag->params, sizeof(*params));
> +
> + return mbuf_dynflag->bitnum;
> +}
> +
> +static int mbuf_dynflag_cmp(const struct rte_mbuf_dynflag *params1,
> + const struct rte_mbuf_dynflag *params2) {
> + if (strcmp(params1->name, params2->name))
> + return -1;
> + if (params1->flags != params2->flags)
> + return -1;
> + return 0;
> +}
> +
> +/* assume tailq is locked */
> +static int
> +__rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag
> *params,
> + unsigned int req)
> +{
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *mbuf_dynflag = NULL;
> + struct rte_tailq_entry *te = NULL;
> + unsigned int bitnum;
> + int ret;
> +
> + if (shm == NULL && init_shared_mem() < 0)
> + return -1;
> +
> + mbuf_dynflag = __mbuf_dynflag_lookup(params->name);
> + if (mbuf_dynflag != NULL) {
> + if (req != UINT_MAX && req != mbuf_dynflag->bitnum) {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + if (mbuf_dynflag_cmp(params, &mbuf_dynflag->params) < 0)
> {
> + rte_errno = EEXIST;
> + return -1;
> + }
> + return mbuf_dynflag->bitnum;
> + }
> +
> + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> + rte_errno = EPERM;
> + return -1;
> + }
> +
> + if (req == UINT_MAX) {
> + if (shm->free_flags == 0) {
> + rte_errno = ENOENT;
> + return -1;
> + }
> + bitnum = rte_bsf64(shm->free_flags);
> + } else {
> + if ((shm->free_flags & (1ULL << req)) == 0) {
> + rte_errno = EBUSY;
> + return -1;
> + }
> + bitnum = req;
> + }
> +
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> +
> + te = rte_zmalloc("MBUF_DYNFLAG_TAILQ_ENTRY", sizeof(*te), 0);
> + if (te == NULL)
> + return -1;
> +
> + mbuf_dynflag = rte_zmalloc("mbuf_dynflag", sizeof(*mbuf_dynflag),
> 0);
> + if (mbuf_dynflag == NULL) {
> + rte_free(te);
> + return -1;
> + }
> +
> + ret = strlcpy(mbuf_dynflag->params.name, params->name,
> + sizeof(mbuf_dynflag->params.name));
> + if (ret < 0 || ret >= (int)sizeof(mbuf_dynflag->params.name)) {
> + rte_free(mbuf_dynflag);
> + rte_free(te);
> + rte_errno = ENAMETOOLONG;
> + return -1;
> + }
> + mbuf_dynflag->bitnum = bitnum;
> + te->data = mbuf_dynflag;
> +
> + TAILQ_INSERT_TAIL(mbuf_dynflag_list, te, next);
> +
> + shm->free_flags &= ~(1ULL << bitnum);
> +
> + RTE_LOG(DEBUG, MBUF, "Registered dynamic flag %s (fl=0x%x) ->
> %u\n",
> + params->name, params->flags, bitnum);
> +
> + return bitnum;
> +}
> +
> +int
> +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> + unsigned int req)
> +{
> + int ret;
> +
> + if (req != UINT_MAX && req >= 64) {
> + rte_errno = EINVAL;
> + return -1;
> + }
> +
> + rte_mcfg_tailq_write_lock();
> + ret = __rte_mbuf_dynflag_register_bitnum(params, req);
> + rte_mcfg_tailq_write_unlock();
> +
> + return ret;
> +}
> +
> +int
> +rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params) {
> + return rte_mbuf_dynflag_register_bitnum(params, UINT_MAX); }
> +
> +void rte_mbuf_dyn_dump(FILE *out)
> +{
> + struct mbuf_dynfield_list *mbuf_dynfield_list;
> + struct mbuf_dynfield_elt *dynfield;
> + struct mbuf_dynflag_list *mbuf_dynflag_list;
> + struct mbuf_dynflag_elt *dynflag;
> + struct rte_tailq_entry *te;
> + size_t i;
> +
> + rte_mcfg_tailq_write_lock();
> + init_shared_mem();
> + fprintf(out, "Reserved fields:\n");
> + mbuf_dynfield_list = RTE_TAILQ_CAST(
> + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> + dynfield = (struct mbuf_dynfield_elt *)te->data;
> + fprintf(out, " name=%s offset=%zd size=%zd align=%zd
> flags=%x\n",
> + dynfield->params.name, dynfield->offset,
> + dynfield->params.size, dynfield->params.align,
> + dynfield->params.flags);
> + }
> + fprintf(out, "Reserved flags:\n");
> + mbuf_dynflag_list = RTE_TAILQ_CAST(
> + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> + dynflag = (struct mbuf_dynflag_elt *)te->data;
> + fprintf(out, " name=%s bitnum=%u flags=%x\n",
> + dynflag->params.name, dynflag->bitnum,
> + dynflag->params.flags);
> + }
> + fprintf(out, "Free space in mbuf (0 = free, value = zone
> alignment):\n");
> + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> + if ((i % 8) == 0)
> + fprintf(out, " %4.4zx: ", i);
> + fprintf(out, "%2.2x%s", shm->free_space[i],
> + (i % 8 != 7) ? " " : "\n");
> + }
> + rte_mcfg_tailq_write_unlock();
> +}
> diff --git a/lib/librte_mbuf/rte_mbuf_dyn.h
> b/lib/librte_mbuf/rte_mbuf_dyn.h new file mode 100644 index
> 000000000..307613c96
> --- /dev/null
> +++ b/lib/librte_mbuf/rte_mbuf_dyn.h
> @@ -0,0 +1,226 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright 2019 6WIND S.A.
> + */
> +
> +#ifndef _RTE_MBUF_DYN_H_
> +#define _RTE_MBUF_DYN_H_
> +
> +/**
> + * @file
> + * RTE Mbuf dynamic fields and flags
> + *
> + * Many features require to store data inside the mbuf. As the room in
> + * mbuf structure is limited, it is not possible to have a field for
> + * each feature. Also, changing fields in the mbuf structure can break
> + * the API or ABI.
> + *
> + * This module addresses this issue, by enabling the dynamic
> + * registration of fields or flags:
> + *
> + * - a dynamic field is a named area in the rte_mbuf structure, with a
> + * given size (>= 1 byte) and alignment constraint.
> + * - a dynamic flag is a named bit in the rte_mbuf structure, stored
> + * in mbuf->ol_flags.
> + *
> + * The typical use case is when a specific offload feature requires to
> + * register a dedicated offload field in the mbuf structure, and adding
> + * a static field or flag is not justified.
> + *
> + * Example of use:
> + *
> + * - A rte_mbuf_dynfield structure is defined, containing the parameters
> + * of the dynamic field to be registered:
> + * const struct rte_mbuf_dynfield rte_dynfield_my_feature = { ... };
> + * - The application initializes the PMD, and asks for this feature
> + * at port initialization by passing DEV_RX_OFFLOAD_MY_FEATURE in
> + * rxconf. This will make the PMD to register the field by calling
> + * rte_mbuf_dynfield_register(&rte_dynfield_my_feature). The PMD
> + * stores the returned offset.
> + * - The application that uses the offload feature also registers
> + * the field to retrieve the same offset.
> + * - When the PMD receives a packet, it can set the field:
> + * *RTE_MBUF_DYNFIELD(m, offset, <type *>) = value;
> + * - In the main loop, the application can retrieve the value with
> + * the same macro.
> + *
> + * To avoid wasting space, the dynamic fields or flags must only be
> + * reserved on demand, when an application asks for the related feature.
> + *
> + * The registration can be done at any moment, but it is not possible
> + * to unregister fields or flags for now.
> + *
> + * A dynamic field can be reserved and used by an application only.
> + * It can for instance be a packet mark.
> + */
> +
> +#include <sys/types.h>
> +/**
> + * Maximum length of the dynamic field or flag string.
> + */
> +#define RTE_MBUF_DYN_NAMESIZE 64
> +
> +/**
> + * Structure describing the parameters of a mbuf dynamic field.
> + */
> +struct rte_mbuf_dynfield {
> + char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the field. */
> + size_t size; /**< The number of bytes to reserve. */
> + size_t align; /**< The alignment constraint (power of 2). */
> + unsigned int flags; /**< Reserved for future use, must be 0. */ };
> +
> +/**
> + * Structure describing the parameters of a mbuf dynamic flag.
> + */
> +struct rte_mbuf_dynflag {
> + char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the dynamic
> flag. */
> + unsigned int flags; /**< Reserved for future use, must be 0. */ };
> +
> +/**
> + * Register space for a dynamic field in the mbuf structure.
> + *
> + * If the field is already registered (same name and parameters), its
> + * offset is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters (name, size,
> + * alignment constraint and flags).
> + * @return
> + * The offset in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, or flags).
> + * - EEXIST: this name is already register with different parameters.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: not enough room in mbuf.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name does not ends with \0.
> + */
> +__rte_experimental
> +int rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params);
> +
> +/**
> + * Register space for a dynamic field in the mbuf structure at offset.
> + *
> + * If the field is already registered (same name, parameters and
> +offset),
> + * the offset is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters (name, size,
> + * alignment constraint and flags).
> + * @param offset
> + * The requested offset. Ignored if SIZE_MAX is passed.
> + * @return
> + * The offset in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, flags, or offset).
> + * - EEXIST: this name is already register with different parameters.
> + * - EBUSY: the requested offset cannot be used.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: not enough room in mbuf.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name does not ends with \0.
> + */
> +__rte_experimental
> +int rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> *params,
> + size_t offset);
> +
> +/**
> + * Lookup for a registered dynamic mbuf field.
> + *
> + * @param name
> + * A string identifying the dynamic field.
> + * @param params
> + * If not NULL, and if the lookup is successful, the structure is
> + * filled with the parameters of the dynamic field.
> + * @return
> + * The offset of this field in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - ENOENT: no dynamic field matches this name.
> + */
> +__rte_experimental
> +int rte_mbuf_dynfield_lookup(const char *name,
> + struct rte_mbuf_dynfield *params);
> +
> +/**
> + * Register a dynamic flag in the mbuf structure.
> + *
> + * If the flag is already registered (same name and parameters), its
> + * bitnum is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters of the dynamic
> + * flag (name and options).
> + * @return
> + * The number of the reserved bit, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, or flags).
> + * - EEXIST: this name is already register with different parameters.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: no more flag available.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE -
> 1.
> + */
> +__rte_experimental
> +int rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params);
> +
> +/**
> + * Register a dynamic flag in the mbuf structure specifying bitnum.
> + *
> + * If the flag is already registered (same name, parameters and
> +bitnum),
> + * the bitnum is returned.
> + *
> + * @param params
> + * A structure containing the requested parameters of the dynamic
> + * flag (name and options).
> + * @param bitnum
> + * The requested bitnum. Ignored if UINT_MAX is passed.
> + * @return
> + * The number of the reserved bit, or -1 on error.
> + * Possible values for rte_errno:
> + * - EINVAL: invalid parameters (size, align, or flags).
> + * - EEXIST: this name is already register with different parameters.
> + * - EBUSY: the requested bitnum cannot be used.
> + * - EPERM: called from a secondary process.
> + * - ENOENT: no more flag available.
> + * - ENOMEM: allocation failure.
> + * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE -
> 1.
> + */
> +__rte_experimental
> +int rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag
> *params,
> + unsigned int bitnum);
> +
> +/**
> + * Lookup for a registered dynamic mbuf flag.
> + *
> + * @param name
> + * A string identifying the dynamic flag.
> + * @param params
> + * If not NULL, and if the lookup is successful, the structure is
> + * filled with the parameters of the dynamic flag.
> + * @return
> + * The offset of this flag in the mbuf structure, or -1 on error.
> + * Possible values for rte_errno:
> + * - ENOENT: no dynamic flag matches this name.
> + */
> +__rte_experimental
> +int rte_mbuf_dynflag_lookup(const char *name,
> + struct rte_mbuf_dynflag *params);
> +
> +/**
> + * Helper macro to access to a dynamic field.
> + */
> +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) +
> +(offset)))
> +
> +/**
> + * Dump the status of dynamic fields and flags.
> + *
> + * @param out
> + * The stream where the status is displayed.
> + */
> +__rte_experimental
> +void rte_mbuf_dyn_dump(FILE *out);
> +
> +/* Placeholder for dynamic fields and flags declarations. */
> +
> +#endif
> diff --git a/lib/librte_mbuf/rte_mbuf_version.map
> b/lib/librte_mbuf/rte_mbuf_version.map
> index 519fead35..9bf5ca37a 100644
> --- a/lib/librte_mbuf/rte_mbuf_version.map
> +++ b/lib/librte_mbuf/rte_mbuf_version.map
> @@ -58,6 +58,13 @@ EXPERIMENTAL {
> global:
>
> rte_mbuf_check;
> + rte_mbuf_dynfield_lookup;
> + rte_mbuf_dynfield_register;
> + rte_mbuf_dynfield_register_offset;
> + rte_mbuf_dynflag_lookup;
> + rte_mbuf_dynflag_register;
> + rte_mbuf_dynflag_register_bitnum;
> + rte_mbuf_dyn_dump;
> rte_pktmbuf_copy;
>
> } DPDK_18.08;
> --
> 2.20.1
On Thu, Oct 24, 2019 at 07:38:15AM +0000, Slava Ovsiienko wrote:
> Hi,
>
> Doc building failed, it seems the rte_mbuf_dynfield_copy() description should be fixed:
>
> ./lib/librte_mbuf/rte_mbuf.h:1694: warning: argument 'm_dst' of command @param is not found in the argument list of rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> ./lib/librte_mbuf/rte_mbuf.h:1694: warning: argument 'm_src' of command @param is not found in the argument list of rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
> ./lib/librte_mbuf/rte_mbuf.h:1694: warning: The following parameters of rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc) are not documented
Thanks for spotting this, I'm adding the fix to the v3.
>
> With best regards,
> Slava
>
> > -----Original Message-----
> > From: dev <dev-bounces@dpdk.org> On Behalf Of Olivier Matz
> > Sent: Thursday, October 17, 2019 17:42
> > To: dev@dpdk.org
> > Cc: Andrew Rybchenko <arybchenko@solarflare.com>; Bruce Richardson
> > <bruce.richardson@intel.com>; Wang, Haiyue <haiyue.wang@intel.com>;
> > Jerin Jacob Kollanukkaran <jerinj@marvell.com>; Wiles, Keith
> > <keith.wiles@intel.com>; Ananyev, Konstantin
> > <konstantin.ananyev@intel.com>; Morten Brørup
> > <mb@smartsharesystems.com>; Stephen Hemminger
> > <stephen@networkplumber.org>; Thomas Monjalon
> > <thomas@monjalon.net>
> > Subject: [dpdk-dev] [PATCH v2] mbuf: support dynamic fields and flags
> >
> > Many features require to store data inside the mbuf. As the room in mbuf
> > structure is limited, it is not possible to have a field for each feature. Also,
> > changing fields in the mbuf structure can break the API or ABI.
> >
> > This commit addresses these issues, by enabling the dynamic registration of
> > fields or flags:
> >
> > - a dynamic field is a named area in the rte_mbuf structure, with a
> > given size (>= 1 byte) and alignment constraint.
> > - a dynamic flag is a named bit in the rte_mbuf structure.
> >
> > The typical use case is a PMD that registers space for an offload feature,
> > when the application requests to enable this feature. As the space in mbuf is
> > limited, the space should only be reserved if it is going to be used (i.e when
> > the application explicitly asks for it).
> >
> > The registration can be done at any moment, but it is not possible to
> > unregister fields or flags for now.
> >
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > Acked-by: Thomas Monjalon <thomas@monjalon.net>
> > ---
> >
> > v2
> >
> > * Rebase on top of master: solve conflict with Stephen's patchset
> > (packet copy)
> > * Add new apis to register a dynamic field/flag at a specific place
> > * Add a dump function (sugg by David)
> > * Enhance field registration function to select the best offset, keeping
> > large aligned zones as much as possible (sugg by Konstantin)
> > * Use a size_t and unsigned int instead of int when relevant
> > (sugg by Konstantin)
> > * Use "uint64_t dynfield1[2]" in mbuf instead of 2 uint64_t fields
> > (sugg by Konstantin)
> > * Remove unused argument in private function (sugg by Konstantin)
> > * Fix and simplify locking (sugg by Konstantin)
> > * Fix minor typo
> >
> > rfc -> v1
> >
> > * Rebase on top of master
> > * Change registration API to use a structure instead of
> > variables, getting rid of #defines (Stephen's comment)
> > * Update flag registration to use a similar API as fields.
> > * Change max name length from 32 to 64 (sugg. by Thomas)
> > * Enhance API documentation (Haiyue's and Andrew's comments)
> > * Add a debug log at registration
> > * Add some words in release note
> > * Did some performance tests (sugg. by Andrew):
> > On my platform, reading a dynamic field takes ~3 cycles more
> > than a static field, and ~2 cycles more for writing.
> >
> > app/test/test_mbuf.c | 145 ++++++-
> > doc/guides/rel_notes/release_19_11.rst | 7 +
> > lib/librte_mbuf/Makefile | 2 +
> > lib/librte_mbuf/meson.build | 6 +-
> > lib/librte_mbuf/rte_mbuf.h | 23 +-
> > lib/librte_mbuf/rte_mbuf_dyn.c | 548 +++++++++++++++++++++++++
> > lib/librte_mbuf/rte_mbuf_dyn.h | 226 ++++++++++
> > lib/librte_mbuf/rte_mbuf_version.map | 7 +
> > 8 files changed, 959 insertions(+), 5 deletions(-) create mode 100644
> > lib/librte_mbuf/rte_mbuf_dyn.c create mode 100644
> > lib/librte_mbuf/rte_mbuf_dyn.h
> >
> > diff --git a/app/test/test_mbuf.c b/app/test/test_mbuf.c index
> > b9c2b2500..01cafad59 100644
> > --- a/app/test/test_mbuf.c
> > +++ b/app/test/test_mbuf.c
> > @@ -28,6 +28,7 @@
> > #include <rte_random.h>
> > #include <rte_cycles.h>
> > #include <rte_malloc.h>
> > +#include <rte_mbuf_dyn.h>
> >
> > #include "test.h"
> >
> > @@ -657,7 +658,6 @@ test_attach_from_different_pool(struct
> > rte_mempool *pktmbuf_pool,
> > rte_pktmbuf_free(clone2);
> > return -1;
> > }
> > -#undef GOTO_FAIL
> >
> > /*
> > * test allocation and free of mbufs
> > @@ -1276,6 +1276,143 @@ test_tx_offload(void)
> > return (v1 == v2) ? 0 : -EINVAL;
> > }
> >
> > +static int
> > +test_mbuf_dyn(struct rte_mempool *pktmbuf_pool) {
> > + const struct rte_mbuf_dynfield dynfield = {
> > + .name = "test-dynfield",
> > + .size = sizeof(uint8_t),
> > + .align = __alignof__(uint8_t),
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield2 = {
> > + .name = "test-dynfield2",
> > + .size = sizeof(uint16_t),
> > + .align = __alignof__(uint16_t),
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield3 = {
> > + .name = "test-dynfield3",
> > + .size = sizeof(uint8_t),
> > + .align = __alignof__(uint8_t),
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield_fail_big = {
> > + .name = "test-dynfield-fail-big",
> > + .size = 256,
> > + .align = 1,
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynfield dynfield_fail_align = {
> > + .name = "test-dynfield-fail-align",
> > + .size = 1,
> > + .align = 3,
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynflag dynflag = {
> > + .name = "test-dynflag",
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynflag dynflag2 = {
> > + .name = "test-dynflag2",
> > + .flags = 0,
> > + };
> > + const struct rte_mbuf_dynflag dynflag3 = {
> > + .name = "test-dynflag3",
> > + .flags = 0,
> > + };
> > + struct rte_mbuf *m = NULL;
> > + int offset, offset2, offset3;
> > + int flag, flag2, flag3;
> > + int ret;
> > +
> > + printf("Test mbuf dynamic fields and flags\n");
> > + rte_mbuf_dyn_dump(stdout);
> > +
> > + offset = rte_mbuf_dynfield_register(&dynfield);
> > + if (offset == -1)
> > + GOTO_FAIL("failed to register dynamic field, offset=%d: %s",
> > + offset, strerror(errno));
> > +
> > + ret = rte_mbuf_dynfield_register(&dynfield);
> > + if (ret != offset)
> > + GOTO_FAIL("failed to lookup dynamic field, ret=%d: %s",
> > + ret, strerror(errno));
> > +
> > + offset2 = rte_mbuf_dynfield_register(&dynfield2);
> > + if (offset2 == -1 || offset2 == offset || (offset2 & 1))
> > + GOTO_FAIL("failed to register dynamic field 2, offset2=%d:
> > %s",
> > + offset2, strerror(errno));
> > +
> > + offset3 = rte_mbuf_dynfield_register_offset(&dynfield3,
> > + offsetof(struct rte_mbuf, dynfield1[1]));
> > + if (offset3 != offsetof(struct rte_mbuf, dynfield1[1]))
> > + GOTO_FAIL("failed to register dynamic field 3, offset=%d:
> > %s",
> > + offset3, strerror(errno));
> > +
> > + printf("dynfield: offset=%d, offset2=%d, offset3=%d\n",
> > + offset, offset2, offset3);
> > +
> > + ret = rte_mbuf_dynfield_register(&dynfield_fail_big);
> > + if (ret != -1)
> > + GOTO_FAIL("dynamic field creation should fail (too big)");
> > +
> > + ret = rte_mbuf_dynfield_register(&dynfield_fail_align);
> > + if (ret != -1)
> > + GOTO_FAIL("dynamic field creation should fail (bad
> > alignment)");
> > +
> > + ret = rte_mbuf_dynfield_register_offset(&dynfield_fail_align,
> > + offsetof(struct rte_mbuf, ol_flags));
> > + if (ret != -1)
> > + GOTO_FAIL("dynamic field creation should fail (not avail)");
> > +
> > + flag = rte_mbuf_dynflag_register(&dynflag);
> > + if (flag == -1)
> > + GOTO_FAIL("failed to register dynamic flag, flag=%d: %s",
> > + flag, strerror(errno));
> > +
> > + ret = rte_mbuf_dynflag_register(&dynflag);
> > + if (ret != flag)
> > + GOTO_FAIL("failed to lookup dynamic flag, ret=%d: %s",
> > + ret, strerror(errno));
> > +
> > + flag2 = rte_mbuf_dynflag_register(&dynflag2);
> > + if (flag2 == -1 || flag2 == flag)
> > + GOTO_FAIL("failed to register dynamic flag 2, flag2=%d: %s",
> > + flag2, strerror(errno));
> > +
> > + flag3 = rte_mbuf_dynflag_register_bitnum(&dynflag3,
> > + rte_bsf64(PKT_LAST_FREE));
> > + if (flag3 != rte_bsf64(PKT_LAST_FREE))
> > + GOTO_FAIL("failed to register dynamic flag 3, flag2=%d: %s",
> > + flag3, strerror(errno));
> > +
> > + printf("dynflag: flag=%d, flag2=%d, flag3=%d\n", flag, flag2, flag3);
> > +
> > + /* set, get dynamic field */
> > + m = rte_pktmbuf_alloc(pktmbuf_pool);
> > + if (m == NULL)
> > + GOTO_FAIL("Cannot allocate mbuf");
> > +
> > + *RTE_MBUF_DYNFIELD(m, offset, uint8_t *) = 1;
> > + if (*RTE_MBUF_DYNFIELD(m, offset, uint8_t *) != 1)
> > + GOTO_FAIL("failed to read dynamic field");
> > + *RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) = 1000;
> > + if (*RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) != 1000)
> > + GOTO_FAIL("failed to read dynamic field");
> > +
> > + /* set a dynamic flag */
> > + m->ol_flags |= (1ULL << flag);
> > +
> > + rte_mbuf_dyn_dump(stdout);
> > + rte_pktmbuf_free(m);
> > + return 0;
> > +fail:
> > + rte_pktmbuf_free(m);
> > + return -1;
> > +}
> > +#undef GOTO_FAIL
> > +
> > static int
> > test_mbuf(void)
> > {
> > @@ -1295,6 +1432,12 @@ test_mbuf(void)
> > goto err;
> > }
> >
> > + /* test registration of dynamic fields and flags */
> > + if (test_mbuf_dyn(pktmbuf_pool) < 0) {
> > + printf("mbuf dynflag test failed\n");
> > + goto err;
> > + }
> > +
> > /* create a specific pktmbuf pool with a priv_size != 0 and no data
> > * room size */
> > pktmbuf_pool2 = rte_pktmbuf_pool_create("test_pktmbuf_pool2",
> > diff --git a/doc/guides/rel_notes/release_19_11.rst
> > b/doc/guides/rel_notes/release_19_11.rst
> > index 85953b962..9e9c94554 100644
> > --- a/doc/guides/rel_notes/release_19_11.rst
> > +++ b/doc/guides/rel_notes/release_19_11.rst
> > @@ -21,6 +21,13 @@ DPDK Release 19.11
> >
> > xdg-open build/doc/html/guides/rel_notes/release_19_11.html
> >
> > +* **Add support of support dynamic fields and flags in mbuf.**
> > +
> > + This new feature adds the ability to dynamically register some room
> > + for a field or a flag in the mbuf structure. This is typically used
> > + for specific offload features, where adding a static field or flag in
> > + the mbuf is not justified.
> > +
> >
> > New Features
> > ------------
> > diff --git a/lib/librte_mbuf/Makefile b/lib/librte_mbuf/Makefile index
> > c8f6d2689..5a9bcee73 100644
> > --- a/lib/librte_mbuf/Makefile
> > +++ b/lib/librte_mbuf/Makefile
> > @@ -17,8 +17,10 @@ LIBABIVER := 5
> >
> > # all source are stored in SRCS-y
> > SRCS-$(CONFIG_RTE_LIBRTE_MBUF) := rte_mbuf.c rte_mbuf_ptype.c
> > rte_mbuf_pool_ops.c
> > +SRCS-$(CONFIG_RTE_LIBRTE_MBUF) += rte_mbuf_dyn.c
> >
> > # install includes
> > SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include := rte_mbuf.h
> > rte_mbuf_ptype.h rte_mbuf_pool_ops.h
> > +SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include += rte_mbuf_dyn.h
> >
> > include $(RTE_SDK)/mk/rte.lib.mk
> > diff --git a/lib/librte_mbuf/meson.build b/lib/librte_mbuf/meson.build index
> > 6cc11ebb4..9137e8f26 100644
> > --- a/lib/librte_mbuf/meson.build
> > +++ b/lib/librte_mbuf/meson.build
> > @@ -2,8 +2,10 @@
> > # Copyright(c) 2017 Intel Corporation
> >
> > version = 5
> > -sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c') -
> > headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h')
> > +sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c',
> > + 'rte_mbuf_dyn.c')
> > +headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h',
> > + 'rte_mbuf_dyn.h')
> > deps += ['mempool']
> >
> > allow_experimental_apis = true
> > diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h index
> > fb0849ac1..5740b1e93 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -198,9 +198,12 @@ extern "C" {
> > #define PKT_RX_OUTER_L4_CKSUM_GOOD (1ULL << 22)
> > #define PKT_RX_OUTER_L4_CKSUM_INVALID ((1ULL << 21) | (1ULL << 22))
> >
> > -/* add new RX flags here */
> > +/* add new RX flags here, don't forget to update PKT_FIRST_FREE */
> >
> > -/* add new TX flags here */
> > +#define PKT_FIRST_FREE (1ULL << 23)
> > +#define PKT_LAST_FREE (1ULL << 39)
> > +
> > +/* add new TX flags here, don't forget to update PKT_LAST_FREE */
> >
> > /**
> > * Indicate that the metadata field in the mbuf is in use.
> > @@ -738,6 +741,7 @@ struct rte_mbuf {
> > */
> > struct rte_mbuf_ext_shared_info *shinfo;
> >
> > + uint64_t dynfield1[2]; /**< Reserved for dynamic fields. */
> > } __rte_cache_aligned;
> >
> > /**
> > @@ -1684,6 +1688,20 @@ rte_pktmbuf_attach_extbuf(struct rte_mbuf *m,
> > void *buf_addr,
> > */
> > #define rte_pktmbuf_detach_extbuf(m) rte_pktmbuf_detach(m)
> >
> > +/**
> > + * Copy dynamic fields from m_src to m_dst.
> > + *
> > + * @param m_dst
> > + * The destination mbuf.
> > + * @param m_src
> > + * The source mbuf.
> > + */
> > +static inline void
> > +rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf
> > +*msrc) {
> > + memcpy(&mdst->dynfield1, msrc->dynfield1, sizeof(mdst-
> > >dynfield1)); }
> > +
> > /* internal */
> > static inline void
> > __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf
> > *msrc) @@ -1695,6 +1713,7 @@ __rte_pktmbuf_copy_hdr(struct rte_mbuf
> > *mdst, const struct rte_mbuf *msrc)
> > mdst->hash = msrc->hash;
> > mdst->packet_type = msrc->packet_type;
> > mdst->timestamp = msrc->timestamp;
> > + rte_mbuf_dynfield_copy(mdst, msrc);
> > }
> >
> > /**
> > diff --git a/lib/librte_mbuf/rte_mbuf_dyn.c
> > b/lib/librte_mbuf/rte_mbuf_dyn.c new file mode 100644 index
> > 000000000..9ef235483
> > --- /dev/null
> > +++ b/lib/librte_mbuf/rte_mbuf_dyn.c
> > @@ -0,0 +1,548 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright 2019 6WIND S.A.
> > + */
> > +
> > +#include <sys/queue.h>
> > +#include <stdint.h>
> > +#include <limits.h>
> > +
> > +#include <rte_common.h>
> > +#include <rte_eal.h>
> > +#include <rte_eal_memconfig.h>
> > +#include <rte_tailq.h>
> > +#include <rte_errno.h>
> > +#include <rte_malloc.h>
> > +#include <rte_string_fns.h>
> > +#include <rte_mbuf.h>
> > +#include <rte_mbuf_dyn.h>
> > +
> > +#define RTE_MBUF_DYN_MZNAME "rte_mbuf_dyn"
> > +
> > +struct mbuf_dynfield_elt {
> > + TAILQ_ENTRY(mbuf_dynfield_elt) next;
> > + struct rte_mbuf_dynfield params;
> > + size_t offset;
> > +};
> > +TAILQ_HEAD(mbuf_dynfield_list, rte_tailq_entry);
> > +
> > +static struct rte_tailq_elem mbuf_dynfield_tailq = {
> > + .name = "RTE_MBUF_DYNFIELD",
> > +};
> > +EAL_REGISTER_TAILQ(mbuf_dynfield_tailq);
> > +
> > +struct mbuf_dynflag_elt {
> > + TAILQ_ENTRY(mbuf_dynflag_elt) next;
> > + struct rte_mbuf_dynflag params;
> > + unsigned int bitnum;
> > +};
> > +TAILQ_HEAD(mbuf_dynflag_list, rte_tailq_entry);
> > +
> > +static struct rte_tailq_elem mbuf_dynflag_tailq = {
> > + .name = "RTE_MBUF_DYNFLAG",
> > +};
> > +EAL_REGISTER_TAILQ(mbuf_dynflag_tailq);
> > +
> > +struct mbuf_dyn_shm {
> > + /**
> > + * For each mbuf byte, free_space[i] != 0 if space is free.
> > + * The value is the size of the biggest aligned element that
> > + * can fit in the zone.
> > + */
> > + uint8_t free_space[sizeof(struct rte_mbuf)];
> > + /** Bitfield of available flags. */
> > + uint64_t free_flags;
> > +};
> > +static struct mbuf_dyn_shm *shm;
> > +
> > +/* Set the value of free_space[] according to the size and alignment of
> > + * the free areas. This helps to select the best place when reserving a
> > + * dynamic field. Assume tailq is locked.
> > + */
> > +static void
> > +process_score(void)
> > +{
> > + size_t off, align, size, i;
> > +
> > + /* first, erase previous info */
> > + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> > + if (shm->free_space[i])
> > + shm->free_space[i] = 1;
> > + }
> > +
> > + for (off = 0; off < sizeof(struct rte_mbuf); off++) {
> > + /* get the size of the free zone */
> > + for (size = 0; shm->free_space[off + size]; size++)
> > + ;
> > + if (size == 0)
> > + continue;
> > +
> > + /* get the alignment of biggest object that can fit in
> > + * the zone at this offset.
> > + */
> > + for (align = 1;
> > + (off % (align << 1)) == 0 && (align << 1) <= size;
> > + align <<= 1)
> > + ;
> > +
> > + /* save it in free_space[] */
> > + for (i = off; i < off + size; i++)
> > + shm->free_space[i] = RTE_MAX(align, shm-
> > >free_space[i]);
> > + }
> > +}
> > +
> > +/* Allocate and initialize the shared memory. Assume tailq is locked */
> > +static int
> > +init_shared_mem(void)
> > +{
> > + const struct rte_memzone *mz;
> > + uint64_t mask;
> > +
> > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> > + mz =
> > rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
> > + sizeof(struct
> > mbuf_dyn_shm),
> > + SOCKET_ID_ANY, 0,
> > + RTE_CACHE_LINE_SIZE);
> > + } else {
> > + mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
> > + }
> > + if (mz == NULL)
> > + return -1;
> > +
> > + shm = mz->addr;
> > +
> > +#define mark_free(field) \
> > + memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
> > + 1, sizeof(((struct rte_mbuf *)0)->field))
> > +
> > + if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
> > + /* init free_space, keep it sync'd with
> > + * rte_mbuf_dynfield_copy().
> > + */
> > + memset(shm, 0, sizeof(*shm));
> > + mark_free(dynfield1);
> > +
> > + /* init free_flags */
> > + for (mask = PKT_FIRST_FREE; mask <= PKT_LAST_FREE; mask
> > <<= 1)
> > + shm->free_flags |= mask;
> > +
> > + process_score();
> > + }
> > +#undef mark_free
> > +
> > + return 0;
> > +}
> > +
> > +/* check if this offset can be used */
> > +static int
> > +check_offset(size_t offset, size_t size, size_t align) {
> > + size_t i;
> > +
> > + if ((offset & (align - 1)) != 0)
> > + return -1;
> > + if (offset + size > sizeof(struct rte_mbuf))
> > + return -1;
> > +
> > + for (i = 0; i < size; i++) {
> > + if (!shm->free_space[i + offset])
> > + return -1;
> > + }
> > +
> > + return 0;
> > +}
> > +
> > +/* assume tailq is locked */
> > +static struct mbuf_dynfield_elt *
> > +__mbuf_dynfield_lookup(const char *name) {
> > + struct mbuf_dynfield_list *mbuf_dynfield_list;
> > + struct mbuf_dynfield_elt *mbuf_dynfield;
> > + struct rte_tailq_entry *te;
> > +
> > + mbuf_dynfield_list = RTE_TAILQ_CAST(
> > + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> > +
> > + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> > + mbuf_dynfield = (struct mbuf_dynfield_elt *)te->data;
> > + if (strcmp(name, mbuf_dynfield->params.name) == 0)
> > + break;
> > + }
> > +
> > + if (te == NULL) {
> > + rte_errno = ENOENT;
> > + return NULL;
> > + }
> > +
> > + return mbuf_dynfield;
> > +}
> > +
> > +int
> > +rte_mbuf_dynfield_lookup(const char *name, struct rte_mbuf_dynfield
> > +*params) {
> > + struct mbuf_dynfield_elt *mbuf_dynfield;
> > +
> > + if (shm == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_read_lock();
> > + mbuf_dynfield = __mbuf_dynfield_lookup(name);
> > + rte_mcfg_tailq_read_unlock();
> > +
> > + if (mbuf_dynfield == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + if (params != NULL)
> > + memcpy(params, &mbuf_dynfield->params,
> > sizeof(*params));
> > +
> > + return mbuf_dynfield->offset;
> > +}
> > +
> > +static int mbuf_dynfield_cmp(const struct rte_mbuf_dynfield *params1,
> > + const struct rte_mbuf_dynfield *params2) {
> > + if (strcmp(params1->name, params2->name))
> > + return -1;
> > + if (params1->size != params2->size)
> > + return -1;
> > + if (params1->align != params2->align)
> > + return -1;
> > + if (params1->flags != params2->flags)
> > + return -1;
> > + return 0;
> > +}
> > +
> > +/* assume tailq is locked */
> > +static int
> > +__rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> > *params,
> > + size_t req)
> > +{
> > + struct mbuf_dynfield_list *mbuf_dynfield_list;
> > + struct mbuf_dynfield_elt *mbuf_dynfield = NULL;
> > + struct rte_tailq_entry *te = NULL;
> > + unsigned int best_zone = UINT_MAX;
> > + size_t i, offset;
> > + int ret;
> > +
> > + if (shm == NULL && init_shared_mem() < 0)
> > + return -1;
> > +
> > + mbuf_dynfield = __mbuf_dynfield_lookup(params->name);
> > + if (mbuf_dynfield != NULL) {
> > + if (req != SIZE_MAX && req != mbuf_dynfield->offset) {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + if (mbuf_dynfield_cmp(params, &mbuf_dynfield->params) <
> > 0) {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + return mbuf_dynfield->offset;
> > + }
> > +
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> > + rte_errno = EPERM;
> > + return -1;
> > + }
> > +
> > + if (req == SIZE_MAX) {
> > + for (offset = 0;
> > + offset < sizeof(struct rte_mbuf);
> > + offset++) {
> > + if (check_offset(offset, params->size,
> > + params->align) == 0 &&
> > + shm->free_space[offset] <
> > best_zone) {
> > + best_zone = shm->free_space[offset];
> > + req = offset;
> > + }
> > + }
> > + if (req == SIZE_MAX) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > + } else {
> > + if (check_offset(req, params->size, params->align) < 0) {
> > + rte_errno = EBUSY;
> > + return -1;
> > + }
> > + }
> > +
> > + offset = req;
> > + mbuf_dynfield_list = RTE_TAILQ_CAST(
> > + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> > +
> > + te = rte_zmalloc("MBUF_DYNFIELD_TAILQ_ENTRY", sizeof(*te), 0);
> > + if (te == NULL)
> > + return -1;
> > +
> > + mbuf_dynfield = rte_zmalloc("mbuf_dynfield",
> > sizeof(*mbuf_dynfield), 0);
> > + if (mbuf_dynfield == NULL) {
> > + rte_free(te);
> > + return -1;
> > + }
> > +
> > + ret = strlcpy(mbuf_dynfield->params.name, params->name,
> > + sizeof(mbuf_dynfield->params.name));
> > + if (ret < 0 || ret >= (int)sizeof(mbuf_dynfield->params.name)) {
> > + rte_errno = ENAMETOOLONG;
> > + rte_free(mbuf_dynfield);
> > + rte_free(te);
> > + return -1;
> > + }
> > + memcpy(&mbuf_dynfield->params, params, sizeof(mbuf_dynfield-
> > >params));
> > + mbuf_dynfield->offset = offset;
> > + te->data = mbuf_dynfield;
> > +
> > + TAILQ_INSERT_TAIL(mbuf_dynfield_list, te, next);
> > +
> > + for (i = offset; i < offset + params->size; i++)
> > + shm->free_space[i] = 0;
> > + process_score();
> > +
> > + RTE_LOG(DEBUG, MBUF, "Registered dynamic field %s (sz=%zu,
> > al=%zu, fl=0x%x) -> %zd\n",
> > + params->name, params->size, params->align, params->flags,
> > + offset);
> > +
> > + return offset;
> > +}
> > +
> > +int
> > +rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> > *params,
> > + size_t req)
> > +{
> > + int ret;
> > +
> > + if (params->size >= sizeof(struct rte_mbuf)) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > + if (!rte_is_power_of_2(params->align)) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > + if (params->flags != 0) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_write_lock();
> > + ret = __rte_mbuf_dynfield_register_offset(params, req);
> > + rte_mcfg_tailq_write_unlock();
> > +
> > + return ret;
> > +}
> > +
> > +int
> > +rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params) {
> > + return rte_mbuf_dynfield_register_offset(params, SIZE_MAX); }
> > +
> > +/* assume tailq is locked */
> > +static struct mbuf_dynflag_elt *
> > +__mbuf_dynflag_lookup(const char *name) {
> > + struct mbuf_dynflag_list *mbuf_dynflag_list;
> > + struct mbuf_dynflag_elt *mbuf_dynflag;
> > + struct rte_tailq_entry *te;
> > +
> > + mbuf_dynflag_list = RTE_TAILQ_CAST(
> > + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> > +
> > + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> > + mbuf_dynflag = (struct mbuf_dynflag_elt *)te->data;
> > + if (strncmp(name, mbuf_dynflag->params.name,
> > + RTE_MBUF_DYN_NAMESIZE) == 0)
> > + break;
> > + }
> > +
> > + if (te == NULL) {
> > + rte_errno = ENOENT;
> > + return NULL;
> > + }
> > +
> > + return mbuf_dynflag;
> > +}
> > +
> > +int
> > +rte_mbuf_dynflag_lookup(const char *name,
> > + struct rte_mbuf_dynflag *params)
> > +{
> > + struct mbuf_dynflag_elt *mbuf_dynflag;
> > +
> > + if (shm == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_read_lock();
> > + mbuf_dynflag = __mbuf_dynflag_lookup(name);
> > + rte_mcfg_tailq_read_unlock();
> > +
> > + if (mbuf_dynflag == NULL) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > +
> > + if (params != NULL)
> > + memcpy(params, &mbuf_dynflag->params, sizeof(*params));
> > +
> > + return mbuf_dynflag->bitnum;
> > +}
> > +
> > +static int mbuf_dynflag_cmp(const struct rte_mbuf_dynflag *params1,
> > + const struct rte_mbuf_dynflag *params2) {
> > + if (strcmp(params1->name, params2->name))
> > + return -1;
> > + if (params1->flags != params2->flags)
> > + return -1;
> > + return 0;
> > +}
> > +
> > +/* assume tailq is locked */
> > +static int
> > +__rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag
> > *params,
> > + unsigned int req)
> > +{
> > + struct mbuf_dynflag_list *mbuf_dynflag_list;
> > + struct mbuf_dynflag_elt *mbuf_dynflag = NULL;
> > + struct rte_tailq_entry *te = NULL;
> > + unsigned int bitnum;
> > + int ret;
> > +
> > + if (shm == NULL && init_shared_mem() < 0)
> > + return -1;
> > +
> > + mbuf_dynflag = __mbuf_dynflag_lookup(params->name);
> > + if (mbuf_dynflag != NULL) {
> > + if (req != UINT_MAX && req != mbuf_dynflag->bitnum) {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + if (mbuf_dynflag_cmp(params, &mbuf_dynflag->params) < 0)
> > {
> > + rte_errno = EEXIST;
> > + return -1;
> > + }
> > + return mbuf_dynflag->bitnum;
> > + }
> > +
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
> > + rte_errno = EPERM;
> > + return -1;
> > + }
> > +
> > + if (req == UINT_MAX) {
> > + if (shm->free_flags == 0) {
> > + rte_errno = ENOENT;
> > + return -1;
> > + }
> > + bitnum = rte_bsf64(shm->free_flags);
> > + } else {
> > + if ((shm->free_flags & (1ULL << req)) == 0) {
> > + rte_errno = EBUSY;
> > + return -1;
> > + }
> > + bitnum = req;
> > + }
> > +
> > + mbuf_dynflag_list = RTE_TAILQ_CAST(
> > + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> > +
> > + te = rte_zmalloc("MBUF_DYNFLAG_TAILQ_ENTRY", sizeof(*te), 0);
> > + if (te == NULL)
> > + return -1;
> > +
> > + mbuf_dynflag = rte_zmalloc("mbuf_dynflag", sizeof(*mbuf_dynflag),
> > 0);
> > + if (mbuf_dynflag == NULL) {
> > + rte_free(te);
> > + return -1;
> > + }
> > +
> > + ret = strlcpy(mbuf_dynflag->params.name, params->name,
> > + sizeof(mbuf_dynflag->params.name));
> > + if (ret < 0 || ret >= (int)sizeof(mbuf_dynflag->params.name)) {
> > + rte_free(mbuf_dynflag);
> > + rte_free(te);
> > + rte_errno = ENAMETOOLONG;
> > + return -1;
> > + }
> > + mbuf_dynflag->bitnum = bitnum;
> > + te->data = mbuf_dynflag;
> > +
> > + TAILQ_INSERT_TAIL(mbuf_dynflag_list, te, next);
> > +
> > + shm->free_flags &= ~(1ULL << bitnum);
> > +
> > + RTE_LOG(DEBUG, MBUF, "Registered dynamic flag %s (fl=0x%x) ->
> > %u\n",
> > + params->name, params->flags, bitnum);
> > +
> > + return bitnum;
> > +}
> > +
> > +int
> > +rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
> > + unsigned int req)
> > +{
> > + int ret;
> > +
> > + if (req != UINT_MAX && req >= 64) {
> > + rte_errno = EINVAL;
> > + return -1;
> > + }
> > +
> > + rte_mcfg_tailq_write_lock();
> > + ret = __rte_mbuf_dynflag_register_bitnum(params, req);
> > + rte_mcfg_tailq_write_unlock();
> > +
> > + return ret;
> > +}
> > +
> > +int
> > +rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params) {
> > + return rte_mbuf_dynflag_register_bitnum(params, UINT_MAX); }
> > +
> > +void rte_mbuf_dyn_dump(FILE *out)
> > +{
> > + struct mbuf_dynfield_list *mbuf_dynfield_list;
> > + struct mbuf_dynfield_elt *dynfield;
> > + struct mbuf_dynflag_list *mbuf_dynflag_list;
> > + struct mbuf_dynflag_elt *dynflag;
> > + struct rte_tailq_entry *te;
> > + size_t i;
> > +
> > + rte_mcfg_tailq_write_lock();
> > + init_shared_mem();
> > + fprintf(out, "Reserved fields:\n");
> > + mbuf_dynfield_list = RTE_TAILQ_CAST(
> > + mbuf_dynfield_tailq.head, mbuf_dynfield_list);
> > + TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
> > + dynfield = (struct mbuf_dynfield_elt *)te->data;
> > + fprintf(out, " name=%s offset=%zd size=%zd align=%zd
> > flags=%x\n",
> > + dynfield->params.name, dynfield->offset,
> > + dynfield->params.size, dynfield->params.align,
> > + dynfield->params.flags);
> > + }
> > + fprintf(out, "Reserved flags:\n");
> > + mbuf_dynflag_list = RTE_TAILQ_CAST(
> > + mbuf_dynflag_tailq.head, mbuf_dynflag_list);
> > + TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
> > + dynflag = (struct mbuf_dynflag_elt *)te->data;
> > + fprintf(out, " name=%s bitnum=%u flags=%x\n",
> > + dynflag->params.name, dynflag->bitnum,
> > + dynflag->params.flags);
> > + }
> > + fprintf(out, "Free space in mbuf (0 = free, value = zone
> > alignment):\n");
> > + for (i = 0; i < sizeof(struct rte_mbuf); i++) {
> > + if ((i % 8) == 0)
> > + fprintf(out, " %4.4zx: ", i);
> > + fprintf(out, "%2.2x%s", shm->free_space[i],
> > + (i % 8 != 7) ? " " : "\n");
> > + }
> > + rte_mcfg_tailq_write_unlock();
> > +}
> > diff --git a/lib/librte_mbuf/rte_mbuf_dyn.h
> > b/lib/librte_mbuf/rte_mbuf_dyn.h new file mode 100644 index
> > 000000000..307613c96
> > --- /dev/null
> > +++ b/lib/librte_mbuf/rte_mbuf_dyn.h
> > @@ -0,0 +1,226 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright 2019 6WIND S.A.
> > + */
> > +
> > +#ifndef _RTE_MBUF_DYN_H_
> > +#define _RTE_MBUF_DYN_H_
> > +
> > +/**
> > + * @file
> > + * RTE Mbuf dynamic fields and flags
> > + *
> > + * Many features require to store data inside the mbuf. As the room in
> > + * mbuf structure is limited, it is not possible to have a field for
> > + * each feature. Also, changing fields in the mbuf structure can break
> > + * the API or ABI.
> > + *
> > + * This module addresses this issue, by enabling the dynamic
> > + * registration of fields or flags:
> > + *
> > + * - a dynamic field is a named area in the rte_mbuf structure, with a
> > + * given size (>= 1 byte) and alignment constraint.
> > + * - a dynamic flag is a named bit in the rte_mbuf structure, stored
> > + * in mbuf->ol_flags.
> > + *
> > + * The typical use case is when a specific offload feature requires to
> > + * register a dedicated offload field in the mbuf structure, and adding
> > + * a static field or flag is not justified.
> > + *
> > + * Example of use:
> > + *
> > + * - A rte_mbuf_dynfield structure is defined, containing the parameters
> > + * of the dynamic field to be registered:
> > + * const struct rte_mbuf_dynfield rte_dynfield_my_feature = { ... };
> > + * - The application initializes the PMD, and asks for this feature
> > + * at port initialization by passing DEV_RX_OFFLOAD_MY_FEATURE in
> > + * rxconf. This will make the PMD to register the field by calling
> > + * rte_mbuf_dynfield_register(&rte_dynfield_my_feature). The PMD
> > + * stores the returned offset.
> > + * - The application that uses the offload feature also registers
> > + * the field to retrieve the same offset.
> > + * - When the PMD receives a packet, it can set the field:
> > + * *RTE_MBUF_DYNFIELD(m, offset, <type *>) = value;
> > + * - In the main loop, the application can retrieve the value with
> > + * the same macro.
> > + *
> > + * To avoid wasting space, the dynamic fields or flags must only be
> > + * reserved on demand, when an application asks for the related feature.
> > + *
> > + * The registration can be done at any moment, but it is not possible
> > + * to unregister fields or flags for now.
> > + *
> > + * A dynamic field can be reserved and used by an application only.
> > + * It can for instance be a packet mark.
> > + */
> > +
> > +#include <sys/types.h>
> > +/**
> > + * Maximum length of the dynamic field or flag string.
> > + */
> > +#define RTE_MBUF_DYN_NAMESIZE 64
> > +
> > +/**
> > + * Structure describing the parameters of a mbuf dynamic field.
> > + */
> > +struct rte_mbuf_dynfield {
> > + char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the field. */
> > + size_t size; /**< The number of bytes to reserve. */
> > + size_t align; /**< The alignment constraint (power of 2). */
> > + unsigned int flags; /**< Reserved for future use, must be 0. */ };
> > +
> > +/**
> > + * Structure describing the parameters of a mbuf dynamic flag.
> > + */
> > +struct rte_mbuf_dynflag {
> > + char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the dynamic
> > flag. */
> > + unsigned int flags; /**< Reserved for future use, must be 0. */ };
> > +
> > +/**
> > + * Register space for a dynamic field in the mbuf structure.
> > + *
> > + * If the field is already registered (same name and parameters), its
> > + * offset is returned.
> > + *
> > + * @param params
> > + * A structure containing the requested parameters (name, size,
> > + * alignment constraint and flags).
> > + * @return
> > + * The offset in the mbuf structure, or -1 on error.
> > + * Possible values for rte_errno:
> > + * - EINVAL: invalid parameters (size, align, or flags).
> > + * - EEXIST: this name is already register with different parameters.
> > + * - EPERM: called from a secondary process.
> > + * - ENOENT: not enough room in mbuf.
> > + * - ENOMEM: allocation failure.
> > + * - ENAMETOOLONG: name does not ends with \0.
> > + */
> > +__rte_experimental
> > +int rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params);
> > +
> > +/**
> > + * Register space for a dynamic field in the mbuf structure at offset.
> > + *
> > + * If the field is already registered (same name, parameters and
> > +offset),
> > + * the offset is returned.
> > + *
> > + * @param params
> > + * A structure containing the requested parameters (name, size,
> > + * alignment constraint and flags).
> > + * @param offset
> > + * The requested offset. Ignored if SIZE_MAX is passed.
> > + * @return
> > + * The offset in the mbuf structure, or -1 on error.
> > + * Possible values for rte_errno:
> > + * - EINVAL: invalid parameters (size, align, flags, or offset).
> > + * - EEXIST: this name is already register with different parameters.
> > + * - EBUSY: the requested offset cannot be used.
> > + * - EPERM: called from a secondary process.
> > + * - ENOENT: not enough room in mbuf.
> > + * - ENOMEM: allocation failure.
> > + * - ENAMETOOLONG: name does not ends with \0.
> > + */
> > +__rte_experimental
> > +int rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield
> > *params,
> > + size_t offset);
> > +
> > +/**
> > + * Lookup for a registered dynamic mbuf field.
> > + *
> > + * @param name
> > + * A string identifying the dynamic field.
> > + * @param params
> > + * If not NULL, and if the lookup is successful, the structure is
> > + * filled with the parameters of the dynamic field.
> > + * @return
> > + * The offset of this field in the mbuf structure, or -1 on error.
> > + * Possible values for rte_errno:
> > + * - ENOENT: no dynamic field matches this name.
> > + */
> > +__rte_experimental
> > +int rte_mbuf_dynfield_lookup(const char *name,
> > + struct rte_mbuf_dynfield *params);
> > +
> > +/**
> > + * Register a dynamic flag in the mbuf structure.
> > + *
> > + * If the flag is already registered (same name and parameters), its
> > + * bitnum is returned.
> > + *
> > + * @param params
> > + * A structure containing the requested parameters of the dynamic
> > + * flag (name and options).
> > + * @return
> > + * The number of the reserved bit, or -1 on error.
> > + * Possible values for rte_errno:
> > + * - EINVAL: invalid parameters (size, align, or flags).
> > + * - EEXIST: this name is already register with different parameters.
> > + * - EPERM: called from a secondary process.
> > + * - ENOENT: no more flag available.
> > + * - ENOMEM: allocation failure.
> > + * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE -
> > 1.
> > + */
> > +__rte_experimental
> > +int rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params);
> > +
> > +/**
> > + * Register a dynamic flag in the mbuf structure specifying bitnum.
> > + *
> > + * If the flag is already registered (same name, parameters and
> > +bitnum),
> > + * the bitnum is returned.
> > + *
> > + * @param params
> > + * A structure containing the requested parameters of the dynamic
> > + * flag (name and options).
> > + * @param bitnum
> > + * The requested bitnum. Ignored if UINT_MAX is passed.
> > + * @return
> > + * The number of the reserved bit, or -1 on error.
> > + * Possible values for rte_errno:
> > + * - EINVAL: invalid parameters (size, align, or flags).
> > + * - EEXIST: this name is already register with different parameters.
> > + * - EBUSY: the requested bitnum cannot be used.
> > + * - EPERM: called from a secondary process.
> > + * - ENOENT: no more flag available.
> > + * - ENOMEM: allocation failure.
> > + * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE -
> > 1.
> > + */
> > +__rte_experimental
> > +int rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag
> > *params,
> > + unsigned int bitnum);
> > +
> > +/**
> > + * Lookup for a registered dynamic mbuf flag.
> > + *
> > + * @param name
> > + * A string identifying the dynamic flag.
> > + * @param params
> > + * If not NULL, and if the lookup is successful, the structure is
> > + * filled with the parameters of the dynamic flag.
> > + * @return
> > + * The offset of this flag in the mbuf structure, or -1 on error.
> > + * Possible values for rte_errno:
> > + * - ENOENT: no dynamic flag matches this name.
> > + */
> > +__rte_experimental
> > +int rte_mbuf_dynflag_lookup(const char *name,
> > + struct rte_mbuf_dynflag *params);
> > +
> > +/**
> > + * Helper macro to access to a dynamic field.
> > + */
> > +#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) +
> > +(offset)))
> > +
> > +/**
> > + * Dump the status of dynamic fields and flags.
> > + *
> > + * @param out
> > + * The stream where the status is displayed.
> > + */
> > +__rte_experimental
> > +void rte_mbuf_dyn_dump(FILE *out);
> > +
> > +/* Placeholder for dynamic fields and flags declarations. */
> > +
> > +#endif
> > diff --git a/lib/librte_mbuf/rte_mbuf_version.map
> > b/lib/librte_mbuf/rte_mbuf_version.map
> > index 519fead35..9bf5ca37a 100644
> > --- a/lib/librte_mbuf/rte_mbuf_version.map
> > +++ b/lib/librte_mbuf/rte_mbuf_version.map
> > @@ -58,6 +58,13 @@ EXPERIMENTAL {
> > global:
> >
> > rte_mbuf_check;
> > + rte_mbuf_dynfield_lookup;
> > + rte_mbuf_dynfield_register;
> > + rte_mbuf_dynfield_register_offset;
> > + rte_mbuf_dynflag_lookup;
> > + rte_mbuf_dynflag_register;
> > + rte_mbuf_dynflag_register_bitnum;
> > + rte_mbuf_dyn_dump;
> > rte_pktmbuf_copy;
> >
> > } DPDK_18.08;
> > --
> > 2.20.1
>
@@ -28,6 +28,7 @@
#include <rte_random.h>
#include <rte_cycles.h>
#include <rte_malloc.h>
+#include <rte_mbuf_dyn.h>
#include "test.h"
@@ -657,7 +658,6 @@ test_attach_from_different_pool(struct rte_mempool *pktmbuf_pool,
rte_pktmbuf_free(clone2);
return -1;
}
-#undef GOTO_FAIL
/*
* test allocation and free of mbufs
@@ -1276,6 +1276,143 @@ test_tx_offload(void)
return (v1 == v2) ? 0 : -EINVAL;
}
+static int
+test_mbuf_dyn(struct rte_mempool *pktmbuf_pool)
+{
+ const struct rte_mbuf_dynfield dynfield = {
+ .name = "test-dynfield",
+ .size = sizeof(uint8_t),
+ .align = __alignof__(uint8_t),
+ .flags = 0,
+ };
+ const struct rte_mbuf_dynfield dynfield2 = {
+ .name = "test-dynfield2",
+ .size = sizeof(uint16_t),
+ .align = __alignof__(uint16_t),
+ .flags = 0,
+ };
+ const struct rte_mbuf_dynfield dynfield3 = {
+ .name = "test-dynfield3",
+ .size = sizeof(uint8_t),
+ .align = __alignof__(uint8_t),
+ .flags = 0,
+ };
+ const struct rte_mbuf_dynfield dynfield_fail_big = {
+ .name = "test-dynfield-fail-big",
+ .size = 256,
+ .align = 1,
+ .flags = 0,
+ };
+ const struct rte_mbuf_dynfield dynfield_fail_align = {
+ .name = "test-dynfield-fail-align",
+ .size = 1,
+ .align = 3,
+ .flags = 0,
+ };
+ const struct rte_mbuf_dynflag dynflag = {
+ .name = "test-dynflag",
+ .flags = 0,
+ };
+ const struct rte_mbuf_dynflag dynflag2 = {
+ .name = "test-dynflag2",
+ .flags = 0,
+ };
+ const struct rte_mbuf_dynflag dynflag3 = {
+ .name = "test-dynflag3",
+ .flags = 0,
+ };
+ struct rte_mbuf *m = NULL;
+ int offset, offset2, offset3;
+ int flag, flag2, flag3;
+ int ret;
+
+ printf("Test mbuf dynamic fields and flags\n");
+ rte_mbuf_dyn_dump(stdout);
+
+ offset = rte_mbuf_dynfield_register(&dynfield);
+ if (offset == -1)
+ GOTO_FAIL("failed to register dynamic field, offset=%d: %s",
+ offset, strerror(errno));
+
+ ret = rte_mbuf_dynfield_register(&dynfield);
+ if (ret != offset)
+ GOTO_FAIL("failed to lookup dynamic field, ret=%d: %s",
+ ret, strerror(errno));
+
+ offset2 = rte_mbuf_dynfield_register(&dynfield2);
+ if (offset2 == -1 || offset2 == offset || (offset2 & 1))
+ GOTO_FAIL("failed to register dynamic field 2, offset2=%d: %s",
+ offset2, strerror(errno));
+
+ offset3 = rte_mbuf_dynfield_register_offset(&dynfield3,
+ offsetof(struct rte_mbuf, dynfield1[1]));
+ if (offset3 != offsetof(struct rte_mbuf, dynfield1[1]))
+ GOTO_FAIL("failed to register dynamic field 3, offset=%d: %s",
+ offset3, strerror(errno));
+
+ printf("dynfield: offset=%d, offset2=%d, offset3=%d\n",
+ offset, offset2, offset3);
+
+ ret = rte_mbuf_dynfield_register(&dynfield_fail_big);
+ if (ret != -1)
+ GOTO_FAIL("dynamic field creation should fail (too big)");
+
+ ret = rte_mbuf_dynfield_register(&dynfield_fail_align);
+ if (ret != -1)
+ GOTO_FAIL("dynamic field creation should fail (bad alignment)");
+
+ ret = rte_mbuf_dynfield_register_offset(&dynfield_fail_align,
+ offsetof(struct rte_mbuf, ol_flags));
+ if (ret != -1)
+ GOTO_FAIL("dynamic field creation should fail (not avail)");
+
+ flag = rte_mbuf_dynflag_register(&dynflag);
+ if (flag == -1)
+ GOTO_FAIL("failed to register dynamic flag, flag=%d: %s",
+ flag, strerror(errno));
+
+ ret = rte_mbuf_dynflag_register(&dynflag);
+ if (ret != flag)
+ GOTO_FAIL("failed to lookup dynamic flag, ret=%d: %s",
+ ret, strerror(errno));
+
+ flag2 = rte_mbuf_dynflag_register(&dynflag2);
+ if (flag2 == -1 || flag2 == flag)
+ GOTO_FAIL("failed to register dynamic flag 2, flag2=%d: %s",
+ flag2, strerror(errno));
+
+ flag3 = rte_mbuf_dynflag_register_bitnum(&dynflag3,
+ rte_bsf64(PKT_LAST_FREE));
+ if (flag3 != rte_bsf64(PKT_LAST_FREE))
+ GOTO_FAIL("failed to register dynamic flag 3, flag2=%d: %s",
+ flag3, strerror(errno));
+
+ printf("dynflag: flag=%d, flag2=%d, flag3=%d\n", flag, flag2, flag3);
+
+ /* set, get dynamic field */
+ m = rte_pktmbuf_alloc(pktmbuf_pool);
+ if (m == NULL)
+ GOTO_FAIL("Cannot allocate mbuf");
+
+ *RTE_MBUF_DYNFIELD(m, offset, uint8_t *) = 1;
+ if (*RTE_MBUF_DYNFIELD(m, offset, uint8_t *) != 1)
+ GOTO_FAIL("failed to read dynamic field");
+ *RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) = 1000;
+ if (*RTE_MBUF_DYNFIELD(m, offset2, uint16_t *) != 1000)
+ GOTO_FAIL("failed to read dynamic field");
+
+ /* set a dynamic flag */
+ m->ol_flags |= (1ULL << flag);
+
+ rte_mbuf_dyn_dump(stdout);
+ rte_pktmbuf_free(m);
+ return 0;
+fail:
+ rte_pktmbuf_free(m);
+ return -1;
+}
+#undef GOTO_FAIL
+
static int
test_mbuf(void)
{
@@ -1295,6 +1432,12 @@ test_mbuf(void)
goto err;
}
+ /* test registration of dynamic fields and flags */
+ if (test_mbuf_dyn(pktmbuf_pool) < 0) {
+ printf("mbuf dynflag test failed\n");
+ goto err;
+ }
+
/* create a specific pktmbuf pool with a priv_size != 0 and no data
* room size */
pktmbuf_pool2 = rte_pktmbuf_pool_create("test_pktmbuf_pool2",
@@ -21,6 +21,13 @@ DPDK Release 19.11
xdg-open build/doc/html/guides/rel_notes/release_19_11.html
+* **Add support of support dynamic fields and flags in mbuf.**
+
+ This new feature adds the ability to dynamically register some room
+ for a field or a flag in the mbuf structure. This is typically used
+ for specific offload features, where adding a static field or flag
+ in the mbuf is not justified.
+
New Features
------------
@@ -17,8 +17,10 @@ LIBABIVER := 5
# all source are stored in SRCS-y
SRCS-$(CONFIG_RTE_LIBRTE_MBUF) := rte_mbuf.c rte_mbuf_ptype.c rte_mbuf_pool_ops.c
+SRCS-$(CONFIG_RTE_LIBRTE_MBUF) += rte_mbuf_dyn.c
# install includes
SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include := rte_mbuf.h rte_mbuf_ptype.h rte_mbuf_pool_ops.h
+SYMLINK-$(CONFIG_RTE_LIBRTE_MBUF)-include += rte_mbuf_dyn.h
include $(RTE_SDK)/mk/rte.lib.mk
@@ -2,8 +2,10 @@
# Copyright(c) 2017 Intel Corporation
version = 5
-sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c')
-headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h')
+sources = files('rte_mbuf.c', 'rte_mbuf_ptype.c', 'rte_mbuf_pool_ops.c',
+ 'rte_mbuf_dyn.c')
+headers = files('rte_mbuf.h', 'rte_mbuf_ptype.h', 'rte_mbuf_pool_ops.h',
+ 'rte_mbuf_dyn.h')
deps += ['mempool']
allow_experimental_apis = true
@@ -198,9 +198,12 @@ extern "C" {
#define PKT_RX_OUTER_L4_CKSUM_GOOD (1ULL << 22)
#define PKT_RX_OUTER_L4_CKSUM_INVALID ((1ULL << 21) | (1ULL << 22))
-/* add new RX flags here */
+/* add new RX flags here, don't forget to update PKT_FIRST_FREE */
-/* add new TX flags here */
+#define PKT_FIRST_FREE (1ULL << 23)
+#define PKT_LAST_FREE (1ULL << 39)
+
+/* add new TX flags here, don't forget to update PKT_LAST_FREE */
/**
* Indicate that the metadata field in the mbuf is in use.
@@ -738,6 +741,7 @@ struct rte_mbuf {
*/
struct rte_mbuf_ext_shared_info *shinfo;
+ uint64_t dynfield1[2]; /**< Reserved for dynamic fields. */
} __rte_cache_aligned;
/**
@@ -1684,6 +1688,20 @@ rte_pktmbuf_attach_extbuf(struct rte_mbuf *m, void *buf_addr,
*/
#define rte_pktmbuf_detach_extbuf(m) rte_pktmbuf_detach(m)
+/**
+ * Copy dynamic fields from m_src to m_dst.
+ *
+ * @param m_dst
+ * The destination mbuf.
+ * @param m_src
+ * The source mbuf.
+ */
+static inline void
+rte_mbuf_dynfield_copy(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
+{
+ memcpy(&mdst->dynfield1, msrc->dynfield1, sizeof(mdst->dynfield1));
+}
+
/* internal */
static inline void
__rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
@@ -1695,6 +1713,7 @@ __rte_pktmbuf_copy_hdr(struct rte_mbuf *mdst, const struct rte_mbuf *msrc)
mdst->hash = msrc->hash;
mdst->packet_type = msrc->packet_type;
mdst->timestamp = msrc->timestamp;
+ rte_mbuf_dynfield_copy(mdst, msrc);
}
/**
new file mode 100644
@@ -0,0 +1,548 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 6WIND S.A.
+ */
+
+#include <sys/queue.h>
+#include <stdint.h>
+#include <limits.h>
+
+#include <rte_common.h>
+#include <rte_eal.h>
+#include <rte_eal_memconfig.h>
+#include <rte_tailq.h>
+#include <rte_errno.h>
+#include <rte_malloc.h>
+#include <rte_string_fns.h>
+#include <rte_mbuf.h>
+#include <rte_mbuf_dyn.h>
+
+#define RTE_MBUF_DYN_MZNAME "rte_mbuf_dyn"
+
+struct mbuf_dynfield_elt {
+ TAILQ_ENTRY(mbuf_dynfield_elt) next;
+ struct rte_mbuf_dynfield params;
+ size_t offset;
+};
+TAILQ_HEAD(mbuf_dynfield_list, rte_tailq_entry);
+
+static struct rte_tailq_elem mbuf_dynfield_tailq = {
+ .name = "RTE_MBUF_DYNFIELD",
+};
+EAL_REGISTER_TAILQ(mbuf_dynfield_tailq);
+
+struct mbuf_dynflag_elt {
+ TAILQ_ENTRY(mbuf_dynflag_elt) next;
+ struct rte_mbuf_dynflag params;
+ unsigned int bitnum;
+};
+TAILQ_HEAD(mbuf_dynflag_list, rte_tailq_entry);
+
+static struct rte_tailq_elem mbuf_dynflag_tailq = {
+ .name = "RTE_MBUF_DYNFLAG",
+};
+EAL_REGISTER_TAILQ(mbuf_dynflag_tailq);
+
+struct mbuf_dyn_shm {
+ /**
+ * For each mbuf byte, free_space[i] != 0 if space is free.
+ * The value is the size of the biggest aligned element that
+ * can fit in the zone.
+ */
+ uint8_t free_space[sizeof(struct rte_mbuf)];
+ /** Bitfield of available flags. */
+ uint64_t free_flags;
+};
+static struct mbuf_dyn_shm *shm;
+
+/* Set the value of free_space[] according to the size and alignment of
+ * the free areas. This helps to select the best place when reserving a
+ * dynamic field. Assume tailq is locked.
+ */
+static void
+process_score(void)
+{
+ size_t off, align, size, i;
+
+ /* first, erase previous info */
+ for (i = 0; i < sizeof(struct rte_mbuf); i++) {
+ if (shm->free_space[i])
+ shm->free_space[i] = 1;
+ }
+
+ for (off = 0; off < sizeof(struct rte_mbuf); off++) {
+ /* get the size of the free zone */
+ for (size = 0; shm->free_space[off + size]; size++)
+ ;
+ if (size == 0)
+ continue;
+
+ /* get the alignment of biggest object that can fit in
+ * the zone at this offset.
+ */
+ for (align = 1;
+ (off % (align << 1)) == 0 && (align << 1) <= size;
+ align <<= 1)
+ ;
+
+ /* save it in free_space[] */
+ for (i = off; i < off + size; i++)
+ shm->free_space[i] = RTE_MAX(align, shm->free_space[i]);
+ }
+}
+
+/* Allocate and initialize the shared memory. Assume tailq is locked */
+static int
+init_shared_mem(void)
+{
+ const struct rte_memzone *mz;
+ uint64_t mask;
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ mz = rte_memzone_reserve_aligned(RTE_MBUF_DYN_MZNAME,
+ sizeof(struct mbuf_dyn_shm),
+ SOCKET_ID_ANY, 0,
+ RTE_CACHE_LINE_SIZE);
+ } else {
+ mz = rte_memzone_lookup(RTE_MBUF_DYN_MZNAME);
+ }
+ if (mz == NULL)
+ return -1;
+
+ shm = mz->addr;
+
+#define mark_free(field) \
+ memset(&shm->free_space[offsetof(struct rte_mbuf, field)], \
+ 1, sizeof(((struct rte_mbuf *)0)->field))
+
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ /* init free_space, keep it sync'd with
+ * rte_mbuf_dynfield_copy().
+ */
+ memset(shm, 0, sizeof(*shm));
+ mark_free(dynfield1);
+
+ /* init free_flags */
+ for (mask = PKT_FIRST_FREE; mask <= PKT_LAST_FREE; mask <<= 1)
+ shm->free_flags |= mask;
+
+ process_score();
+ }
+#undef mark_free
+
+ return 0;
+}
+
+/* check if this offset can be used */
+static int
+check_offset(size_t offset, size_t size, size_t align)
+{
+ size_t i;
+
+ if ((offset & (align - 1)) != 0)
+ return -1;
+ if (offset + size > sizeof(struct rte_mbuf))
+ return -1;
+
+ for (i = 0; i < size; i++) {
+ if (!shm->free_space[i + offset])
+ return -1;
+ }
+
+ return 0;
+}
+
+/* assume tailq is locked */
+static struct mbuf_dynfield_elt *
+__mbuf_dynfield_lookup(const char *name)
+{
+ struct mbuf_dynfield_list *mbuf_dynfield_list;
+ struct mbuf_dynfield_elt *mbuf_dynfield;
+ struct rte_tailq_entry *te;
+
+ mbuf_dynfield_list = RTE_TAILQ_CAST(
+ mbuf_dynfield_tailq.head, mbuf_dynfield_list);
+
+ TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
+ mbuf_dynfield = (struct mbuf_dynfield_elt *)te->data;
+ if (strcmp(name, mbuf_dynfield->params.name) == 0)
+ break;
+ }
+
+ if (te == NULL) {
+ rte_errno = ENOENT;
+ return NULL;
+ }
+
+ return mbuf_dynfield;
+}
+
+int
+rte_mbuf_dynfield_lookup(const char *name, struct rte_mbuf_dynfield *params)
+{
+ struct mbuf_dynfield_elt *mbuf_dynfield;
+
+ if (shm == NULL) {
+ rte_errno = ENOENT;
+ return -1;
+ }
+
+ rte_mcfg_tailq_read_lock();
+ mbuf_dynfield = __mbuf_dynfield_lookup(name);
+ rte_mcfg_tailq_read_unlock();
+
+ if (mbuf_dynfield == NULL) {
+ rte_errno = ENOENT;
+ return -1;
+ }
+
+ if (params != NULL)
+ memcpy(params, &mbuf_dynfield->params, sizeof(*params));
+
+ return mbuf_dynfield->offset;
+}
+
+static int mbuf_dynfield_cmp(const struct rte_mbuf_dynfield *params1,
+ const struct rte_mbuf_dynfield *params2)
+{
+ if (strcmp(params1->name, params2->name))
+ return -1;
+ if (params1->size != params2->size)
+ return -1;
+ if (params1->align != params2->align)
+ return -1;
+ if (params1->flags != params2->flags)
+ return -1;
+ return 0;
+}
+
+/* assume tailq is locked */
+static int
+__rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params,
+ size_t req)
+{
+ struct mbuf_dynfield_list *mbuf_dynfield_list;
+ struct mbuf_dynfield_elt *mbuf_dynfield = NULL;
+ struct rte_tailq_entry *te = NULL;
+ unsigned int best_zone = UINT_MAX;
+ size_t i, offset;
+ int ret;
+
+ if (shm == NULL && init_shared_mem() < 0)
+ return -1;
+
+ mbuf_dynfield = __mbuf_dynfield_lookup(params->name);
+ if (mbuf_dynfield != NULL) {
+ if (req != SIZE_MAX && req != mbuf_dynfield->offset) {
+ rte_errno = EEXIST;
+ return -1;
+ }
+ if (mbuf_dynfield_cmp(params, &mbuf_dynfield->params) < 0) {
+ rte_errno = EEXIST;
+ return -1;
+ }
+ return mbuf_dynfield->offset;
+ }
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ rte_errno = EPERM;
+ return -1;
+ }
+
+ if (req == SIZE_MAX) {
+ for (offset = 0;
+ offset < sizeof(struct rte_mbuf);
+ offset++) {
+ if (check_offset(offset, params->size,
+ params->align) == 0 &&
+ shm->free_space[offset] < best_zone) {
+ best_zone = shm->free_space[offset];
+ req = offset;
+ }
+ }
+ if (req == SIZE_MAX) {
+ rte_errno = ENOENT;
+ return -1;
+ }
+ } else {
+ if (check_offset(req, params->size, params->align) < 0) {
+ rte_errno = EBUSY;
+ return -1;
+ }
+ }
+
+ offset = req;
+ mbuf_dynfield_list = RTE_TAILQ_CAST(
+ mbuf_dynfield_tailq.head, mbuf_dynfield_list);
+
+ te = rte_zmalloc("MBUF_DYNFIELD_TAILQ_ENTRY", sizeof(*te), 0);
+ if (te == NULL)
+ return -1;
+
+ mbuf_dynfield = rte_zmalloc("mbuf_dynfield", sizeof(*mbuf_dynfield), 0);
+ if (mbuf_dynfield == NULL) {
+ rte_free(te);
+ return -1;
+ }
+
+ ret = strlcpy(mbuf_dynfield->params.name, params->name,
+ sizeof(mbuf_dynfield->params.name));
+ if (ret < 0 || ret >= (int)sizeof(mbuf_dynfield->params.name)) {
+ rte_errno = ENAMETOOLONG;
+ rte_free(mbuf_dynfield);
+ rte_free(te);
+ return -1;
+ }
+ memcpy(&mbuf_dynfield->params, params, sizeof(mbuf_dynfield->params));
+ mbuf_dynfield->offset = offset;
+ te->data = mbuf_dynfield;
+
+ TAILQ_INSERT_TAIL(mbuf_dynfield_list, te, next);
+
+ for (i = offset; i < offset + params->size; i++)
+ shm->free_space[i] = 0;
+ process_score();
+
+ RTE_LOG(DEBUG, MBUF, "Registered dynamic field %s (sz=%zu, al=%zu, fl=0x%x) -> %zd\n",
+ params->name, params->size, params->align, params->flags,
+ offset);
+
+ return offset;
+}
+
+int
+rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params,
+ size_t req)
+{
+ int ret;
+
+ if (params->size >= sizeof(struct rte_mbuf)) {
+ rte_errno = EINVAL;
+ return -1;
+ }
+ if (!rte_is_power_of_2(params->align)) {
+ rte_errno = EINVAL;
+ return -1;
+ }
+ if (params->flags != 0) {
+ rte_errno = EINVAL;
+ return -1;
+ }
+
+ rte_mcfg_tailq_write_lock();
+ ret = __rte_mbuf_dynfield_register_offset(params, req);
+ rte_mcfg_tailq_write_unlock();
+
+ return ret;
+}
+
+int
+rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params)
+{
+ return rte_mbuf_dynfield_register_offset(params, SIZE_MAX);
+}
+
+/* assume tailq is locked */
+static struct mbuf_dynflag_elt *
+__mbuf_dynflag_lookup(const char *name)
+{
+ struct mbuf_dynflag_list *mbuf_dynflag_list;
+ struct mbuf_dynflag_elt *mbuf_dynflag;
+ struct rte_tailq_entry *te;
+
+ mbuf_dynflag_list = RTE_TAILQ_CAST(
+ mbuf_dynflag_tailq.head, mbuf_dynflag_list);
+
+ TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
+ mbuf_dynflag = (struct mbuf_dynflag_elt *)te->data;
+ if (strncmp(name, mbuf_dynflag->params.name,
+ RTE_MBUF_DYN_NAMESIZE) == 0)
+ break;
+ }
+
+ if (te == NULL) {
+ rte_errno = ENOENT;
+ return NULL;
+ }
+
+ return mbuf_dynflag;
+}
+
+int
+rte_mbuf_dynflag_lookup(const char *name,
+ struct rte_mbuf_dynflag *params)
+{
+ struct mbuf_dynflag_elt *mbuf_dynflag;
+
+ if (shm == NULL) {
+ rte_errno = ENOENT;
+ return -1;
+ }
+
+ rte_mcfg_tailq_read_lock();
+ mbuf_dynflag = __mbuf_dynflag_lookup(name);
+ rte_mcfg_tailq_read_unlock();
+
+ if (mbuf_dynflag == NULL) {
+ rte_errno = ENOENT;
+ return -1;
+ }
+
+ if (params != NULL)
+ memcpy(params, &mbuf_dynflag->params, sizeof(*params));
+
+ return mbuf_dynflag->bitnum;
+}
+
+static int mbuf_dynflag_cmp(const struct rte_mbuf_dynflag *params1,
+ const struct rte_mbuf_dynflag *params2)
+{
+ if (strcmp(params1->name, params2->name))
+ return -1;
+ if (params1->flags != params2->flags)
+ return -1;
+ return 0;
+}
+
+/* assume tailq is locked */
+static int
+__rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
+ unsigned int req)
+{
+ struct mbuf_dynflag_list *mbuf_dynflag_list;
+ struct mbuf_dynflag_elt *mbuf_dynflag = NULL;
+ struct rte_tailq_entry *te = NULL;
+ unsigned int bitnum;
+ int ret;
+
+ if (shm == NULL && init_shared_mem() < 0)
+ return -1;
+
+ mbuf_dynflag = __mbuf_dynflag_lookup(params->name);
+ if (mbuf_dynflag != NULL) {
+ if (req != UINT_MAX && req != mbuf_dynflag->bitnum) {
+ rte_errno = EEXIST;
+ return -1;
+ }
+ if (mbuf_dynflag_cmp(params, &mbuf_dynflag->params) < 0) {
+ rte_errno = EEXIST;
+ return -1;
+ }
+ return mbuf_dynflag->bitnum;
+ }
+
+ if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
+ rte_errno = EPERM;
+ return -1;
+ }
+
+ if (req == UINT_MAX) {
+ if (shm->free_flags == 0) {
+ rte_errno = ENOENT;
+ return -1;
+ }
+ bitnum = rte_bsf64(shm->free_flags);
+ } else {
+ if ((shm->free_flags & (1ULL << req)) == 0) {
+ rte_errno = EBUSY;
+ return -1;
+ }
+ bitnum = req;
+ }
+
+ mbuf_dynflag_list = RTE_TAILQ_CAST(
+ mbuf_dynflag_tailq.head, mbuf_dynflag_list);
+
+ te = rte_zmalloc("MBUF_DYNFLAG_TAILQ_ENTRY", sizeof(*te), 0);
+ if (te == NULL)
+ return -1;
+
+ mbuf_dynflag = rte_zmalloc("mbuf_dynflag", sizeof(*mbuf_dynflag), 0);
+ if (mbuf_dynflag == NULL) {
+ rte_free(te);
+ return -1;
+ }
+
+ ret = strlcpy(mbuf_dynflag->params.name, params->name,
+ sizeof(mbuf_dynflag->params.name));
+ if (ret < 0 || ret >= (int)sizeof(mbuf_dynflag->params.name)) {
+ rte_free(mbuf_dynflag);
+ rte_free(te);
+ rte_errno = ENAMETOOLONG;
+ return -1;
+ }
+ mbuf_dynflag->bitnum = bitnum;
+ te->data = mbuf_dynflag;
+
+ TAILQ_INSERT_TAIL(mbuf_dynflag_list, te, next);
+
+ shm->free_flags &= ~(1ULL << bitnum);
+
+ RTE_LOG(DEBUG, MBUF, "Registered dynamic flag %s (fl=0x%x) -> %u\n",
+ params->name, params->flags, bitnum);
+
+ return bitnum;
+}
+
+int
+rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
+ unsigned int req)
+{
+ int ret;
+
+ if (req != UINT_MAX && req >= 64) {
+ rte_errno = EINVAL;
+ return -1;
+ }
+
+ rte_mcfg_tailq_write_lock();
+ ret = __rte_mbuf_dynflag_register_bitnum(params, req);
+ rte_mcfg_tailq_write_unlock();
+
+ return ret;
+}
+
+int
+rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params)
+{
+ return rte_mbuf_dynflag_register_bitnum(params, UINT_MAX);
+}
+
+void rte_mbuf_dyn_dump(FILE *out)
+{
+ struct mbuf_dynfield_list *mbuf_dynfield_list;
+ struct mbuf_dynfield_elt *dynfield;
+ struct mbuf_dynflag_list *mbuf_dynflag_list;
+ struct mbuf_dynflag_elt *dynflag;
+ struct rte_tailq_entry *te;
+ size_t i;
+
+ rte_mcfg_tailq_write_lock();
+ init_shared_mem();
+ fprintf(out, "Reserved fields:\n");
+ mbuf_dynfield_list = RTE_TAILQ_CAST(
+ mbuf_dynfield_tailq.head, mbuf_dynfield_list);
+ TAILQ_FOREACH(te, mbuf_dynfield_list, next) {
+ dynfield = (struct mbuf_dynfield_elt *)te->data;
+ fprintf(out, " name=%s offset=%zd size=%zd align=%zd flags=%x\n",
+ dynfield->params.name, dynfield->offset,
+ dynfield->params.size, dynfield->params.align,
+ dynfield->params.flags);
+ }
+ fprintf(out, "Reserved flags:\n");
+ mbuf_dynflag_list = RTE_TAILQ_CAST(
+ mbuf_dynflag_tailq.head, mbuf_dynflag_list);
+ TAILQ_FOREACH(te, mbuf_dynflag_list, next) {
+ dynflag = (struct mbuf_dynflag_elt *)te->data;
+ fprintf(out, " name=%s bitnum=%u flags=%x\n",
+ dynflag->params.name, dynflag->bitnum,
+ dynflag->params.flags);
+ }
+ fprintf(out, "Free space in mbuf (0 = free, value = zone alignment):\n");
+ for (i = 0; i < sizeof(struct rte_mbuf); i++) {
+ if ((i % 8) == 0)
+ fprintf(out, " %4.4zx: ", i);
+ fprintf(out, "%2.2x%s", shm->free_space[i],
+ (i % 8 != 7) ? " " : "\n");
+ }
+ rte_mcfg_tailq_write_unlock();
+}
new file mode 100644
@@ -0,0 +1,226 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2019 6WIND S.A.
+ */
+
+#ifndef _RTE_MBUF_DYN_H_
+#define _RTE_MBUF_DYN_H_
+
+/**
+ * @file
+ * RTE Mbuf dynamic fields and flags
+ *
+ * Many features require to store data inside the mbuf. As the room in
+ * mbuf structure is limited, it is not possible to have a field for
+ * each feature. Also, changing fields in the mbuf structure can break
+ * the API or ABI.
+ *
+ * This module addresses this issue, by enabling the dynamic
+ * registration of fields or flags:
+ *
+ * - a dynamic field is a named area in the rte_mbuf structure, with a
+ * given size (>= 1 byte) and alignment constraint.
+ * - a dynamic flag is a named bit in the rte_mbuf structure, stored
+ * in mbuf->ol_flags.
+ *
+ * The typical use case is when a specific offload feature requires to
+ * register a dedicated offload field in the mbuf structure, and adding
+ * a static field or flag is not justified.
+ *
+ * Example of use:
+ *
+ * - A rte_mbuf_dynfield structure is defined, containing the parameters
+ * of the dynamic field to be registered:
+ * const struct rte_mbuf_dynfield rte_dynfield_my_feature = { ... };
+ * - The application initializes the PMD, and asks for this feature
+ * at port initialization by passing DEV_RX_OFFLOAD_MY_FEATURE in
+ * rxconf. This will make the PMD to register the field by calling
+ * rte_mbuf_dynfield_register(&rte_dynfield_my_feature). The PMD
+ * stores the returned offset.
+ * - The application that uses the offload feature also registers
+ * the field to retrieve the same offset.
+ * - When the PMD receives a packet, it can set the field:
+ * *RTE_MBUF_DYNFIELD(m, offset, <type *>) = value;
+ * - In the main loop, the application can retrieve the value with
+ * the same macro.
+ *
+ * To avoid wasting space, the dynamic fields or flags must only be
+ * reserved on demand, when an application asks for the related feature.
+ *
+ * The registration can be done at any moment, but it is not possible
+ * to unregister fields or flags for now.
+ *
+ * A dynamic field can be reserved and used by an application only.
+ * It can for instance be a packet mark.
+ */
+
+#include <sys/types.h>
+/**
+ * Maximum length of the dynamic field or flag string.
+ */
+#define RTE_MBUF_DYN_NAMESIZE 64
+
+/**
+ * Structure describing the parameters of a mbuf dynamic field.
+ */
+struct rte_mbuf_dynfield {
+ char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the field. */
+ size_t size; /**< The number of bytes to reserve. */
+ size_t align; /**< The alignment constraint (power of 2). */
+ unsigned int flags; /**< Reserved for future use, must be 0. */
+};
+
+/**
+ * Structure describing the parameters of a mbuf dynamic flag.
+ */
+struct rte_mbuf_dynflag {
+ char name[RTE_MBUF_DYN_NAMESIZE]; /**< Name of the dynamic flag. */
+ unsigned int flags; /**< Reserved for future use, must be 0. */
+};
+
+/**
+ * Register space for a dynamic field in the mbuf structure.
+ *
+ * If the field is already registered (same name and parameters), its
+ * offset is returned.
+ *
+ * @param params
+ * A structure containing the requested parameters (name, size,
+ * alignment constraint and flags).
+ * @return
+ * The offset in the mbuf structure, or -1 on error.
+ * Possible values for rte_errno:
+ * - EINVAL: invalid parameters (size, align, or flags).
+ * - EEXIST: this name is already register with different parameters.
+ * - EPERM: called from a secondary process.
+ * - ENOENT: not enough room in mbuf.
+ * - ENOMEM: allocation failure.
+ * - ENAMETOOLONG: name does not ends with \0.
+ */
+__rte_experimental
+int rte_mbuf_dynfield_register(const struct rte_mbuf_dynfield *params);
+
+/**
+ * Register space for a dynamic field in the mbuf structure at offset.
+ *
+ * If the field is already registered (same name, parameters and offset),
+ * the offset is returned.
+ *
+ * @param params
+ * A structure containing the requested parameters (name, size,
+ * alignment constraint and flags).
+ * @param offset
+ * The requested offset. Ignored if SIZE_MAX is passed.
+ * @return
+ * The offset in the mbuf structure, or -1 on error.
+ * Possible values for rte_errno:
+ * - EINVAL: invalid parameters (size, align, flags, or offset).
+ * - EEXIST: this name is already register with different parameters.
+ * - EBUSY: the requested offset cannot be used.
+ * - EPERM: called from a secondary process.
+ * - ENOENT: not enough room in mbuf.
+ * - ENOMEM: allocation failure.
+ * - ENAMETOOLONG: name does not ends with \0.
+ */
+__rte_experimental
+int rte_mbuf_dynfield_register_offset(const struct rte_mbuf_dynfield *params,
+ size_t offset);
+
+/**
+ * Lookup for a registered dynamic mbuf field.
+ *
+ * @param name
+ * A string identifying the dynamic field.
+ * @param params
+ * If not NULL, and if the lookup is successful, the structure is
+ * filled with the parameters of the dynamic field.
+ * @return
+ * The offset of this field in the mbuf structure, or -1 on error.
+ * Possible values for rte_errno:
+ * - ENOENT: no dynamic field matches this name.
+ */
+__rte_experimental
+int rte_mbuf_dynfield_lookup(const char *name,
+ struct rte_mbuf_dynfield *params);
+
+/**
+ * Register a dynamic flag in the mbuf structure.
+ *
+ * If the flag is already registered (same name and parameters), its
+ * bitnum is returned.
+ *
+ * @param params
+ * A structure containing the requested parameters of the dynamic
+ * flag (name and options).
+ * @return
+ * The number of the reserved bit, or -1 on error.
+ * Possible values for rte_errno:
+ * - EINVAL: invalid parameters (size, align, or flags).
+ * - EEXIST: this name is already register with different parameters.
+ * - EPERM: called from a secondary process.
+ * - ENOENT: no more flag available.
+ * - ENOMEM: allocation failure.
+ * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE - 1.
+ */
+__rte_experimental
+int rte_mbuf_dynflag_register(const struct rte_mbuf_dynflag *params);
+
+/**
+ * Register a dynamic flag in the mbuf structure specifying bitnum.
+ *
+ * If the flag is already registered (same name, parameters and bitnum),
+ * the bitnum is returned.
+ *
+ * @param params
+ * A structure containing the requested parameters of the dynamic
+ * flag (name and options).
+ * @param bitnum
+ * The requested bitnum. Ignored if UINT_MAX is passed.
+ * @return
+ * The number of the reserved bit, or -1 on error.
+ * Possible values for rte_errno:
+ * - EINVAL: invalid parameters (size, align, or flags).
+ * - EEXIST: this name is already register with different parameters.
+ * - EBUSY: the requested bitnum cannot be used.
+ * - EPERM: called from a secondary process.
+ * - ENOENT: no more flag available.
+ * - ENOMEM: allocation failure.
+ * - ENAMETOOLONG: name is longer than RTE_MBUF_DYN_NAMESIZE - 1.
+ */
+__rte_experimental
+int rte_mbuf_dynflag_register_bitnum(const struct rte_mbuf_dynflag *params,
+ unsigned int bitnum);
+
+/**
+ * Lookup for a registered dynamic mbuf flag.
+ *
+ * @param name
+ * A string identifying the dynamic flag.
+ * @param params
+ * If not NULL, and if the lookup is successful, the structure is
+ * filled with the parameters of the dynamic flag.
+ * @return
+ * The offset of this flag in the mbuf structure, or -1 on error.
+ * Possible values for rte_errno:
+ * - ENOENT: no dynamic flag matches this name.
+ */
+__rte_experimental
+int rte_mbuf_dynflag_lookup(const char *name,
+ struct rte_mbuf_dynflag *params);
+
+/**
+ * Helper macro to access to a dynamic field.
+ */
+#define RTE_MBUF_DYNFIELD(m, offset, type) ((type)((uintptr_t)(m) + (offset)))
+
+/**
+ * Dump the status of dynamic fields and flags.
+ *
+ * @param out
+ * The stream where the status is displayed.
+ */
+__rte_experimental
+void rte_mbuf_dyn_dump(FILE *out);
+
+/* Placeholder for dynamic fields and flags declarations. */
+
+#endif
@@ -58,6 +58,13 @@ EXPERIMENTAL {
global:
rte_mbuf_check;
+ rte_mbuf_dynfield_lookup;
+ rte_mbuf_dynfield_register;
+ rte_mbuf_dynfield_register_offset;
+ rte_mbuf_dynflag_lookup;
+ rte_mbuf_dynflag_register;
+ rte_mbuf_dynflag_register_bitnum;
+ rte_mbuf_dyn_dump;
rte_pktmbuf_copy;
} DPDK_18.08;