[dpdk-dev] [RFC PATCH 0/4]: Implement module information export

[Neil Horman]

On Wed, May 04, 2016 at 10:24:18AM +0200, David Marchand wrote:
> On Tue, May 3, 2016 at 1:57 PM, Neil Horman <nhorman at tuxdriver.com> wrote:
> > On Tue, Apr 26, 2016 at 01:39:47PM -0400, Neil Horman wrote:
> >> Hey-
> >>       So a few days ago we were reviewing Davids patch series to introduce the
> >> abiilty to dump hardware support from pmd DSO's in a human readable format.
> >> That effort encountered some problems, most notably the fact that stripping a
> >> DSO removed the required information that the proposed tool keyed off, as well
> >> as the need to dead reckon offsets between symbols that may not be constant
> >> (dependent on architecture).
> >>
> >>       I was going to start looking into the possibility of creating a modinfo
> >> section in a simmilar fashion to what kernel modules do in linux or BSD.  I
> >> decided to propose this solution instead though, because the kernel style
> >> solution requires a significant amount of infrastructure that I think we can
> >> maybe avoid maintaining, if we accept some minor caviats
> >>
> >> To do this We emit a set of well known marker symbols for each DSO that an
> >> external application can search for (in this case I called them
> >> this_pmd_driver<n>, where n is a counter macro).  These marker symbols are
> >> n is a counter macro).  These marker symbols are exported by PMDs for
> >> external access.  External tools can then access these symbols via the
> >> dlopen/dlsym api (or via elfutils libraries)
> >>
> >> The symbols above alias the rte_driver struct for each PMD, and the external
> >> application can then interrogate the registered driver information.
> >>
> >> I also add a pointer to the pci id table struct for each PMD so that we can
> >> export hardware support.
> >>
> >> This approach has a few pros and cons:
> >>
> >> pros:
> >> 1) Its simple, and doesn't require extra infrastructure to implement.  E.g. we
> >> don't need a new tool to extract driver information and emit the C code to build
> >> the binary data for the special section, nor do we need a custom linker script
> >> to link said special section in place
> >>
> >> 2) Its stable.  Because the marker symbols are explicitly exported, this
> >> approach is resilient against stripping.
> >>
> >> cons:
> >> 1) It creates an artifact in that PMD_REGISTER_DRIVER has to be used in one
> >> compilation unit per DSO.  As an example em and igb effectively merge two
> >> drivers into one DSO, and the uses of PMD_REGISTER_DRIVER occur in two separate
> >> C files for the same single linked DSO.  Because of the use of the __COUNTER__
> >> macro we get multiple definitions of the same marker symbols.
> >>
> >> I would make the argument that the downside of the above artifact isn't that big
> >> a deal.  Traditionally in other projects a unit like a module (or DSO in our
> >> case) only ever codifies a single driver (e.g. module_init() in the linux kernel
> >> is only ever used once per built module).  If we have code like igb/em that
> >> shares some core code, we should build the shared code to object files and link
> >> them twice, once to an em.so pmd and again to an igb.so pmd.
> >>
> >> But regardless, I thought I would propose this to see what you all thought of
> >> it.
> >>
> >> FWIW, heres sample output of the pmdinfo tool from this series probing the
> >> librte_pmd_ena.so module:
> >>
> >> [nhorman at hmsreliant dpdk]$ ./build/app/pmdinfo
> >> ~/git/dpdk/build/lib/librte_pmd_ena.so
> >> PMD 0 Information:
> >> Driver Name: ena_driver
> >> Driver Type: PCI
> >> |====================PCI Table========================|
> >> | VENDOR ID | DEVICE ID | SUBVENDOR ID | SUBDEVICE ID |
> >> |-----------------------------------------------------|
> >> |       1d0f|       ec20|          ffff|          ffff|
> >> |       1d0f|       ec21|          ffff|          ffff|
> >> |-----------------------------------------------------|
> >>
> >>
> >>
> >>
> > Ping, thoughts here?
> 
> - This implementation does not support binaries, so it is not suitable
> for people who don't want dso, this is partially why I used bfd rather
> than just dlopen.
If you're statically linking an application, you should know what hardware you
support already.  Its going to be very hard, if not impossible to develop a
robust solution that works with static binaries (the prior solutions don't work
consistently with static binaries either).  I really think the static solution
needs to just be built into the application (i.e. the application needs to add a
command line option to dump out the pci id's that are registered).

> - The name of the tool "pmdinfo" is likely to cause problems, I would
> say we need to prefix t with dpdk.
Why?

> - How does it behave if we strip the dsos ?
I described this above, its invariant to stripping, because the symbols for each
pmd are explicitly exported, so strip doesn't touch the symbols that pmdinfo
keys off of.

> - Using __COUNTER__ seeems a bit tricky to me, can't this cause misalignments ?
I'm not sure what you mean here. __COUNTER__ expands to a numerical
string that I concatinate with a fixed string, to provide pmdinfo with a set of
well known symbols to look for.  What does misalignment have to do with anything
here?

> - The tool output format is not script friendly from my pov.
Don't think it really needs to be script friendly, it was meant to provide human
readable output, but script friendly output can be added easily enough if you
want.

Neil

> 
> 
> -- 
> David Marchand
>