[dpdk-dev] [PATCHv6 0/7] Implement pmd hardware support exports

[Neil Horman]

On Tue, May 31, 2016 at 09:57:41AM -0400, Neil Horman wrote:
> Hey all-
> 	So heres attempt number 2 at a method for exporting PMD hardware support
> information.  As we discussed previously, the consensus seems to be that pmd
> information should be:
> 
> 1) Able to be interrogated on any ELF binary (application binary or individual
> DSO)
> 2) Equally functional on statically linked applications or on DSO's
> 3) Resilient to symbol stripping
> 4) Script friendly
> 5) Show kernel dependencies
> 6) List driver options
> 7) Show driver name
> 8) Offer human readable output
> 9) Show DPDK version
> 10) Show driver version
> 11) Allow for expansion
> 12) Not place additional build environment dependencies on an application
> 
> I added item 12 myself, because I don't think its reasonable to require
> applications to use a specific linker script to get hardware support information
> (which precludes the use of a special .modinfo section like the kernel uses)
> 
> However, we still can use some tricks from the kernel to make this work.  In
> this approach, what I've done is the following:
> 
> A) Modified the driver registration macro to also define a variable:
> this_pmd_name<n>= "name"
> 
> Based on the unique name string pointed to by the above variable, we can
> query for an arbitrary number of other symbols following the pattern:
> <name>_<tag>
> 
> Where tag is some well known identifier for information we wish to export
> 
> B) Added a utility called pmdinfogen.  This utility is not meant for general use,
> but rather is used by the dpdk build environment itself when compiling pmds.
> for each object that uses the PMD_REGISTER_DRIVER macro, this utiity is run.  It
> searches for the symbols in (A), and using those, extracts the hardware support
> info, and module name from the object, using that to produce a new c file
> containing a single variable in the following format:
> 
> static const char <name>[] __attribute__((used)) = "PMD_DRIVER_INFO=<json>";
> 
> The <name> is arbitrary, as its static and not referenced.  The relevant bit is
> the string value assigned to it.  The <json> is a json encoded string of the
> extracted hardware support information pmdinfo found for the corresponding
> object.  This C file is suitable for compilation and relocatable linking back
> into the parent object file.  The result of this operation is that the object
> string table now contains a string that will not e removed by stripping, whos
> leading text (PMD_DRIVER_INFO) can be easily searched for at any time weather
> the symbol referring to it is stripped or not.
> 
> C) Added a utilty called pmdinfo.py.  This python script, searches the
> string table of the .rodata section of any provided ELF file looking for the
> PMD_DRIVER_INFO prefix on a string.  When found, it will interpret the remainder
> of the string as json, and output the hardware support for that ELF file (if
> any).
> 
> 
> This approach ticks most of the above boxes:
> 1) Impervious to stripping
> 2) Works on static and dynamic binaries
> 3) Human and script friendly
> 4) Allows for expansion
> 
> Because of (4) the other items should be pretty easy to implement, as its just a
> matter of modifying the macros to export the info, pmdinfo to encode it to json,
> and pmd_hw_support.py to read it.  I'm not adding them now, as theres alot of
> rote work to do to get some of them in place (e.g. DPDK has no current global
> VERSION macro, drivers don't have a consistent version scheme, command line
> strings have to be built for each driver, etc).  But once this is accepted,
> those items can be done as time allows and should be fairly easy to implement.
> 
> Change Notes:
> v2)
>  * Made the export macros a bit easier to expand
>  * Added a macro to optionally export command line strings
>  * Renamed utilties to be more appropriate
>    (pmdinfo -> pmdinfogen, pmd_hw_support.py -> pmdinfo.py)
>  * Added search capabilities to pmdinfo.py so that we search for libraries
>    linked using DT_NEEDINFO entries.  We search DT_RUNPATH, LD_LIBRARY_PATH,
>    /usr/lib and /lib
>  * Added serch abilities if the specified binary to pmdinfo isn't found, we
>    search LD_LIBRARY_PATH, /usr/lib and /lib
>  * Added an option to pmdinfo.py to pretty-print hardware support info using the
>    pci.ids database
> 
> v3)
>  * Made table mode printing the default mode
>  * Added a default path to the pci.ids file
>  * Modifed pmdinfo to use python rather than python3
>  * Cleaned up some changelog entries
>  * Added an export for RTE_EAL_PMD_PATH
>  * Added a plugin option to pmdinfo to scan for autoloaded DSO's
> 
> v4)
>  * Modified the operation of the -p option. As much as I don't like implying
> that autoloaded pmds are guaranteed to be there at run time, I'm having a hard
> time seeing how we can avoid specifying the application file to scan for the
> autoload directory.  Without it we can't determine which library the user means
> in a multiversion installation
>  * Cleaned up the help text
>  * Added a rule for an install target for pmdinfo
>  * Guarded against some tracebacks in pmdinfo
>  * Use DT_NEEDED entries to get versioned libraries in -p mode
>  * Fixed traceback that occurs on lack of input arguments
>  * Fixed some erroneous macro usage in drivers that aren't in the default build
> 
> v5)
>  * Added a dpdk- prefix to pmdinfo symlink
>  * Renamed build announcement to PMDINFOGEN/BUILD
>  * Removed erroneous true statement from makefile
>  * Removed duplicate rte.hostapp.mk makefile
>  * Fixed some whitespace
>  * Whitespace fixups
>  * Fixed makefile if; then style
>  * Renamed module string C file
>  * Removed duplicate rte_pci_id definition
>  * Clarified macro names
>  * Removed PMD type attribute
>  * Fixed tools usage for 32 bit arches
>  * Removed some unused code
>  * Added a few comments
> 
> v6)
>  * Added some programming guide documentation
>  * Reformatted python script with pep8
> 
> Signed-off-by: Neil Horman <nhorman at tuxdriver.com>
> CC: Bruce Richardson <bruce.richardson at intel.com>
> CC: Thomas Monjalon <thomas.monjalon at 6wind.com>
> CC: Stephen Hemminger <stephen at networkplumber.org>
> CC: Panu Matilainen <pmatilai at redhat.com>
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
Ping, feedback here?
Neil