[dpdk-dev] [PATCH v7 11/21] eal/soc: implement probing of drivers

[Shreyansh Jain]

On Thursday 10 November 2016 09:00 AM, Jianbo Liu wrote:
> On 28 October 2016 at 20:26, Shreyansh Jain <shreyansh.jain at nxp.com> wrote:
>> Each SoC PMD registers a set of callback for scanning its own bus/infra and
>> matching devices to drivers when probe is called.
>> This patch introduces the infra for calls to SoC scan on rte_eal_soc_init()
>> and match on rte_eal_soc_probe().
>>
>> Patch also adds test case for scan and probe.
>>
>> Signed-off-by: Jan Viktorin <viktorin at rehivetech.com>
>> Signed-off-by: Shreyansh Jain <shreyansh.jain at nxp.com>
>> Signed-off-by: Hemant Agrawal <hemant.agrawal at nxp.com>
>> --
>> v4:
>>  - Update test_soc for descriptive test function names
>>  - Comments over test functions
>>  - devinit and devuninint --> probe/remove
>>  - RTE_VERIFY at some places
>> ---
>>  app/test/test_soc.c                             | 205 ++++++++++++++++++++++-
>>  lib/librte_eal/bsdapp/eal/rte_eal_version.map   |   4 +
>>  lib/librte_eal/common/eal_common_soc.c          | 213 +++++++++++++++++++++++-
>>  lib/librte_eal/common/include/rte_soc.h         |  75 ++++++++-
>>  lib/librte_eal/linuxapp/eal/eal.c               |   5 +
>>  lib/librte_eal/linuxapp/eal/eal_soc.c           |  21 ++-
>>  lib/librte_eal/linuxapp/eal/rte_eal_version.map |   4 +
>>  7 files changed, 519 insertions(+), 8 deletions(-)
>>
> ......
>
>>  /**
>> + * SoC device scan callback, called from rte_eal_soc_init.
>> + * For various SoC, the bus on which devices are attached maynot be compliant
>> + * to a standard platform (or platform bus itself). In which case, extra
>> + * steps are implemented by PMD to scan over the bus and add devices to SoC
>> + * device list.
>> + */
>> +typedef void (soc_scan_t)(void);
>
> I'm still not sure about the purpose of soc_scan, and how to use it.

For each device to be used by DPDK, which cannot be scanned/identified 
using the existing PCI/VDEV methods (sysfs/bus/pci), 'soc_scan_t' 
provides a way for driver to make those devices part of device lists.

Ideally, 'scan' is not a function of a driver. It is a bus function - 
which is missing in this case.

> If it's for each driver, it should at least struct rte_soc_driver * as
> its parameter.

Its for each driver - assuming that each non-PCI driver which implements 
it knows how to find devices which it can control (for example, special 
area in sysfs, or even platform bus).

> If it's for each bus, why it is in rte_soc_driver?

Short answer - lack of a better place. It should be in dev.h probably 
(rte_device/driver) but it would look out of place (as that represents 
PCI devices also which cannot implement it - all PCI devices are scanned 
in one go irrespective of driver)

> I know you will implement bus driver in the future, but we need to
> make it clear for current simplified implementation.

Current implementation makes only a single assumption - that rather than 
relying on EAL for identifying devices (as being done now), next best 
option in existing framework (driver) should have control of finding 
devices.

This is primarily to make the SoC work parallel to PCI implementation 
without much top-down changes in EAL.

Bus model, improvises it by moving this implementation a little above in 
hierarchy - in rte_bus<-rte_driver<-PMD.

I understand your apprehension - 'driver-scanning-for-devices' is indeed 
not correct real world analogy. It is just a place holder for enabling 
those drivers/PMDs which cannot work in absence of the right model.
And that is still work in progress.

>
>> +
>> +/**
>> + * Custom device<=>driver match callback for SoC
>> + * Unlike PCI, SoC devices don't have a fixed definition of device
>> + * identification. PMDs can implement a specific matching function in which
>> + * driver and device objects are provided to perform custom match.
>> + */
>> +typedef int (soc_match_t)(struct rte_soc_driver *, struct rte_soc_device *);
>> +
>> +/**
>>   * A structure describing a SoC driver.
>>   */
>>  struct rte_soc_driver {
>> @@ -104,6 +120,8 @@ struct rte_soc_driver {
>>         struct rte_driver driver;          /**< Inherit core driver. */
>>         soc_probe_t *probe;                /**< Device probe */
>>         soc_remove_t *remove;              /**< Device remove */
>> +       soc_scan_t *scan_fn;               /**< Callback for scanning SoC bus*/
>> +       soc_match_t *match_fn;             /**< Callback to match dev<->drv */
>>         const struct rte_soc_id *id_table; /**< ID table, NULL terminated */
>>  };
>>
>> @@ -146,12 +164,63 @@ rte_eal_compare_soc_addr(const struct rte_soc_addr *a0,
>>  }
>>
>>  /**
>> + * Default function for matching the Soc driver with device. Each driver can
>> + * either use this function or define their own soc matching function.
>> + * This function relies on the compatible string extracted from sysfs. But,
>> + * a SoC might have different way of identifying its devices. Such SoC can
>> + * override match_fn.
>> + *
>> + * @return
>> + *      0 on success
>> + *     -1 when no match found
>> +  */
>> +int
>> +rte_eal_soc_match_compat(struct rte_soc_driver *drv,
>> +                        struct rte_soc_device *dev);
>> +
>> +/**
>> + * Probe SoC devices for registered drivers.
>> + *
>> + * @return
>> + *     0 on success
>> + *     !0 in case of any failure in probe
>> + */
>> +int rte_eal_soc_probe(void);
>> +
>> +/**
>> + * Probe the single SoC device.
>> + */
>> +int rte_eal_soc_probe_one(const struct rte_soc_addr *addr);
>> +
>> +/**
>> + * Close the single SoC device.
>> + *
>> + * Scan the SoC devices and find the SoC device specified by the SoC
>> + * address, then call the remove() function for registered driver
>> + * that has a matching entry in its id_table for discovered device.
>> + *
>> + * @param addr
>> + *     The SoC address to close.
>> + * @return
>> + *   - 0 on success.
>> + *   - Negative on error.
>> + */
>> +int rte_eal_soc_detach(const struct rte_soc_addr *addr);
>> +
>> +/**
>>   * Dump discovered SoC devices.
>> + *
>> + * @param f
>> + *     File to dump device info in.
>>   */
>>  void rte_eal_soc_dump(FILE *f);
>>
>>  /**
>>   * Register a SoC driver.
>> + *
>> + * @param driver
>> + *     Object for SoC driver to register
>> + * @return void
>>   */
>>  void rte_eal_soc_register(struct rte_soc_driver *driver);
>>
>> @@ -167,6 +236,10 @@ RTE_PMD_EXPORT_NAME(nm, __COUNTER__)
>>
>>  /**
>>   * Unregister a SoC driver.
>> + *
>> + * @param driver
>> + *     Object for SoC driver to unregister
>> + * @return void
>>   */
>>  void rte_eal_soc_unregister(struct rte_soc_driver *driver);
>>
>> diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
>> index 098ba02..bd775f3 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal.c
>> @@ -70,6 +70,7 @@
>>  #include <rte_cpuflags.h>
>>  #include <rte_interrupts.h>
>>  #include <rte_pci.h>
>> +#include <rte_soc.h>
>>  #include <rte_dev.h>
>>  #include <rte_devargs.h>
>>  #include <rte_common.h>
>> @@ -890,6 +891,10 @@ rte_eal_init(int argc, char **argv)
>>         if (rte_eal_pci_probe())
>>                 rte_panic("Cannot probe PCI\n");
>>
>> +       /* Probe & Initialize SoC devices */
>> +       if (rte_eal_soc_probe())
>> +               rte_panic("Cannot probe SoC\n");
>> +
>>         rte_eal_mcfg_complete();
>>
>>         return fctret;
>> diff --git a/lib/librte_eal/linuxapp/eal/eal_soc.c b/lib/librte_eal/linuxapp/eal/eal_soc.c
>> index 04848b9..3929a76 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal_soc.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal_soc.c
>> @@ -44,13 +44,28 @@
>>  #include <rte_log.h>
>>  #include <rte_soc.h>
>>
>> -#include "eal_internal_cfg.h"
>> -#include "eal_filesystem.h"
>> -#include "eal_private.h"
>> +#include <eal_internal_cfg.h>
>> +#include <eal_filesystem.h>
>> +#include <eal_private.h>
>>
>>  /* Init the SoC EAL subsystem */
>>  int
>>  rte_eal_soc_init(void)
>>  {
>> +       struct rte_soc_driver *drv;
>> +
>> +       /* SoC is disabled by default */
>> +       if (!internal_config.enable_soc)
>> +               return 0;
>> +
>> +       /* For each registered driver, call their scan routine to perform any
>> +        * custom scan for devices (for example, custom buses)
>> +        */
>> +       TAILQ_FOREACH(drv, &soc_driver_list, next) {
>> +               RTE_VERIFY(drv->scan_fn);
>> +               drv->scan_fn();
>> +               /* Ignore all errors from this */
>> +       }
>> +
>>         return 0;
>>  }
>> diff --git a/lib/librte_eal/linuxapp/eal/rte_eal_version.map b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>> index 0155025..c28e093 100644
>> --- a/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>> +++ b/lib/librte_eal/linuxapp/eal/rte_eal_version.map
>> @@ -175,7 +175,11 @@ DPDK_16.11 {
>>         rte_eal_dev_attach;
>>         rte_eal_dev_detach;
>>         rte_eal_map_resource;
>> +       rte_eal_soc_detach;
>>         rte_eal_soc_dump;
>> +       rte_eal_soc_match;
>> +       rte_eal_soc_probe;
>> +       rte_eal_soc_probe_one;
>>         rte_eal_soc_register;
>>         rte_eal_soc_unregister;
>>         rte_eal_unmap_resource;
>> --
>> 2.7.4
>>
>


-- 
-
Shreyansh