[dpdk-dev] [PATCH v7 00/21] Introduce SoC device/driver framework for EAL

Shreyansh Jain shreyansh.jain at nxp.com
Fri Oct 28 14:26:17 CEST 2016


Introduction:
=============

This patch set is direct derivative of Jan's original series [1],[2].

 - This version is based on master HEAD (ca41215)
 
 - In this, I am merging the series [11] back. It was initially part
   of this set but I had split considering that those changes in PCI
   were good standalone as well. But, 1) not much feedback was avail-
   able and 2) this patchset is a use-case for those patches making
   it easier to review. Just like what Jan had intended in original
   series.

 - SoC support is not enabled by default. It needs the 'enable-soc' toggle
   on command line. This is primarily because this patchset is still
   experimental and we would like to keep it isolated from non-SoC ops.
   Though, it does impact the ABI.

Aim:
====

As of now EAL is primarly focused on PCI initialization/probing.

 rte_eal_init()
  |- rte_eal_pci_init(): Find PCI devices from sysfs
  |- ...
  |- rte_eal_memzone_init()
  |- ...
  `- rte_eal_pci_probe(): Driver<=>Device initialization

This patchset introduces SoC framework which would enable SoC drivers and
drivers to be plugged into EAL, very similar to how PCI drivers/devices are
done today.

This is a stripped down version of PCI framework which allows the SoC PMDs
to implement their own routines for detecting devices and linking devices to
drivers.

1) Changes to EAL
 rte_eal_init()
  |- rte_eal_pci_init(): Find PCI devices from sysfs
  |- rte_eal_soc_init(): Calls PMDs->scan_fn
  |- ...
  |- rte_eal_memzone_init()
  |- ...
  |- rte_eal_pci_probe(): Driver<=>Device initialization, PMD->devinit()
  `- rte_eal_soc_probe(): Calls PMDs->match_fn and PMDs->devinit();

2) New device/driver structures:
  - rte_soc_driver (inheriting rte_driver)
  - rte_soc_device (inheriting rte_device)
  - rte_eth_dev and eth_driver embedded rte_soc_device and rte_soc_driver,
    respectively.

3) The SoC PMDs need to:
 - define rte_soc_driver with necessary scan and match callbacks
 - Register themselves using DRIVER_REGISTER_SOC()
 - Implement respective bus scanning in the scan callbacks to add necessary
   devices to SoC device list
 - Implement necessary eth_dev_init/uninint for ethernet instances

4) Design considerations that are same as PCI:
 - SoC initialization is being done through rte_eal_init(), just after PCI
   initialization is done.
 - As in case of PCI, probe is done after rte_eal_pci_probe() to link the
   devices detected with the drivers registered.
 - Device attach/detach functions are available and have been designed on
   the lines of PCI framework.
 - PMDs register using DRIVER_REGISTER_SOC, very similar to
   DRIVER_REGISTER_PCI for PCI devices.
 - Linked list of SoC driver and devices exists independent of the other
   driver/device list, but inheriting rte_driver/rte_driver, these are
   also part of a global list.

5) Design considerations that are different from PCI:
 - Each driver implements its own scan and match function. PCI uses the BDF
   format to read the device from sysfs, but this _may_not_ be a case for a
   SoC ethernet device.
   = This is an important change from initial proposal by Jan in [2].
   Unlike his attempt to use /sys/bus/platform, this patch relies on the
   PMD to detect the devices. This is because SoC may require specific or
   additional info for device detection. Further, SoC may have embedded
   devices/MACs which require initialization which cannot be covered
   through sysfs parsing.
   `-> Point (6) below is a side note to above.
   = PCI based PMDs rely on EAL's capability to detect devices. This
   proposal puts the onus on PMD to detect devices, add to soc_device_list
   and wait for Probe. Matching, of device<=>driver is again PMD's
   callback.

6) Adding default scan and match helpers for PMDs
 - The design warrrants the PMDs implement their own scan of devices
   on bus, and match routines for probe implementation.
   This patch introduces helpers which can be used by PMDs for scan of
   the platform bus and matching devices against the compatible string
   extracted from the scan.
 - Intention is to make it easier to integrate known SoC which expose
   platform bus compliant information (compat, sys/bus/platform...).
 - PMDs which have deviations from this standard model can implement and
   hook their bus scanning and probe match callbacks while registering
   driver.

Patchset Overview:
==================
 - Patches 0001~0004 are from [11] - moving some PCI specific functions
   and definitions to non-PCI area.
 - Patches 0005~0008 introduce the base infrastructure and test case
 - Patch 0009 is for command line support for no-soc, on lines of no-pci
 - Patch 0010 enables EAL to handle SoC type devices
 - Patch 0011 adds support for scan and probe callbacks and updates the test
   framework with relevant test case.
 - Patch 0012~0014 enable device argument, driver specific flags and
   interrupt handling related basic infra. Subsequent patches build up on
   them.
 - Patch 0015~0016 add support for default function which PMDs can use for
   scanning platform bus. These functions are optional and need to be hooked
   to by PMDs.
 - Patch 0017~0019 makes changes to PCI as well as ethdev code to remove
   assumption that eth_driver is a PCI driver.
 - Patch 0020 adds necessary ethdev probe/remove functions for PMDs to use
 - Patch 0021 adds support for SoC driver/devices, along with probe/remove
   functions for Cryptodev devices.

Future/Pending Changes:
=======================
- Device whitelisting/blacklist still relies on command line '-b' and '-c'
  which are internally implemented using OPT_PCI_BLACKLIST/OPT_PCI_WHITELIST.
  This needs to be changed to a generic form - OPT_DEV_*LIST - probably.
- No cryptodriver currently uses SoC framework - probably a example driver
  can be created to demonstrate usage.
- test case for enable-soc command line parameter
- This patch impacts a couple of ABIs (rte_device/driver) and thus require
  a patch for bump of libraries (if need be) and documentation.

 [1] http://dpdk.org/ml/archives/dev/2016-January/030915.html
 [2] http://www.dpdk.org/ml/archives/dev/2016-May/038486.html
 [3] http://dpdk.org/ml/archives/dev/2016-August/045707.html
 [4] http://dpdk.org/ml/archives/dev/2016-May/038948.html
 [5] http://dpdk.org/ml/archives/dev/2016-May/038953.html
 [6] http://dpdk.org/ml/archives/dev/2016-May/038487.html
 [7] http://dpdk.org/ml/archives/dev/2016-May/038488.html
 [8] http://dpdk.org/ml/archives/dev/2016-May/038489.html
 [9] http://dpdk.org/ml/archives/dev/2016-May/038491.html
[10] http://dpdk.org/ml/archives/dev/2016-September/046256.html
[11] http://dpdk.org/ml/archives/dev/2016-October/048915.html

Changes since v6;
 - Fix patch rebase over HEAD (ca41215)

Changes since v5:
 - fix devinit/devuninit name change; it was in wrong patch
 - rebased over HEAD (ca41215)
 - Update to pending section of coverletter

Changes since v4:
 - change name of rte_eal_soc_scan function name to
   rte_eal_soc_scan_platform_bus. This still remains a helper function.
 - Fix comments over scan and match functions

Changes since v3:
 - rebasing over HEAD (fed622df tag: v16.11-rc1)
 - Add support for default scan function; PMD can use this for
   scanning on platform bus.
 - Support for kernel driver bind/unbind, numa and DMA from
   Jan's original patches.
 - SoC is disabled by default. '--enable-soc' command line parameter
   enables it. doc updated.
 - Updated testcase function names and comments
 - Map file addition alphabetically ordered
 - Patch author corrected

Changes since v2:
 - Rebasing over rte_driver/device patchset v9 [10]
 - Added cryptodev support for SoC
 - Default match function for SoC device<=>Driver
 - Some variables renamed to reflect 'drv' rather than 'dr'

Change since v1 [2]:
 - Removed patch 1-5 which were for generalizing some PCI specific routines
   into EAL. These patches are good-to-have but not directly linked to SoC
   and hence would be proposed separately.
 - Removed support for sysfs parsing (patches 6~9)
 - Rebasing over the recent (v8) version of rte_driver/device patchset
 - Rebasing over master (16.07)
 - Changes to various map file to change API intro to 16.11 from 16.07

Jan Viktorin (19):
  eal: generalize PCI kernel driver enum to EAL
  eal: generalize PCI map/unmap resource to EAL
  eal/linux: generalize PCI kernel unbinding driver to EAL
  eal/linux: generalize PCI kernel driver extraction to EAL
  eal: define container macro
  eal/soc: introduce very essential SoC infra definitions
  eal/soc: add SoC PMD register/unregister logic
  eal/soc: implement SoC device list and dump
  eal: introduce command line enable SoC option
  eal/soc: init SoC infra from EAL
  eal/soc: extend and utilize devargs
  eal/soc: add drv_flags
  eal/soc: add intr_handle
  eal/soc: add default scan for Soc devices
  eal/soc: additional features for SoC
  ether: utilize container_of for pci_drv
  ether: verify we copy info from a PCI device
  ether: extract function eth_dev_get_intr_handle
  ether: introduce ethernet dev probe remove

Shreyansh Jain (2):
  eal/soc: implement probing of drivers
  eal/crypto: Support rte_soc_driver/device for cryptodev

 app/test/Makefile                               |   1 +
 app/test/test_soc.c                             | 404 +++++++++++++++++++
 doc/guides/testpmd_app_ug/run_app.rst           |   4 +
 lib/librte_cryptodev/rte_cryptodev.c            | 122 +++++-
 lib/librte_cryptodev/rte_cryptodev.h            |   3 +
 lib/librte_cryptodev/rte_cryptodev_pmd.h        |  18 +-
 lib/librte_cryptodev/rte_cryptodev_version.map  |   2 +
 lib/librte_eal/bsdapp/eal/Makefile              |   1 +
 lib/librte_eal/bsdapp/eal/eal.c                 |  18 +
 lib/librte_eal/bsdapp/eal/eal_pci.c             |   6 +-
 lib/librte_eal/bsdapp/eal/eal_soc.c             |  46 +++
 lib/librte_eal/bsdapp/eal/rte_eal_version.map   |  11 +
 lib/librte_eal/common/Makefile                  |   2 +-
 lib/librte_eal/common/eal_common_dev.c          |  66 ++-
 lib/librte_eal/common/eal_common_devargs.c      |  17 +
 lib/librte_eal/common/eal_common_options.c      |   5 +
 lib/librte_eal/common/eal_common_pci.c          |  39 --
 lib/librte_eal/common/eal_common_pci_uio.c      |  16 +-
 lib/librte_eal/common/eal_common_soc.c          | 368 +++++++++++++++++
 lib/librte_eal/common/eal_internal_cfg.h        |   1 +
 lib/librte_eal/common/eal_options.h             |   2 +
 lib/librte_eal/common/eal_private.h             |  64 +++
 lib/librte_eal/common/include/rte_common.h      |  18 +
 lib/librte_eal/common/include/rte_dev.h         |  44 ++
 lib/librte_eal/common/include/rte_devargs.h     |   8 +
 lib/librte_eal/common/include/rte_pci.h         |  41 --
 lib/librte_eal/common/include/rte_soc.h         | 322 +++++++++++++++
 lib/librte_eal/linuxapp/eal/Makefile            |   2 +
 lib/librte_eal/linuxapp/eal/eal.c               |  63 +++
 lib/librte_eal/linuxapp/eal/eal_pci.c           |  62 +--
 lib/librte_eal/linuxapp/eal/eal_pci_uio.c       |   2 +-
 lib/librte_eal/linuxapp/eal/eal_pci_vfio.c      |   5 +-
 lib/librte_eal/linuxapp/eal/eal_soc.c           | 515 ++++++++++++++++++++++++
 lib/librte_eal/linuxapp/eal/rte_eal_version.map |  11 +
 lib/librte_ether/rte_ethdev.c                   | 166 +++++++-
 lib/librte_ether/rte_ethdev.h                   |  33 +-
 36 files changed, 2343 insertions(+), 165 deletions(-)
 create mode 100644 app/test/test_soc.c
 create mode 100644 lib/librte_eal/bsdapp/eal/eal_soc.c
 create mode 100644 lib/librte_eal/common/eal_common_soc.c
 create mode 100644 lib/librte_eal/common/include/rte_soc.h
 create mode 100644 lib/librte_eal/linuxapp/eal/eal_soc.c

-- 
2.7.4



More information about the dev mailing list