[dpdk-dev] [PATCH v2 09/62] net/cnxk: add build infra and common probe
Nithin Dabilpuram
nithind1988 at gmail.com
Wed Jun 9 12:45:39 CEST 2021
On Wed, Jun 09, 2021 at 09:38:04AM +0800, Huisong Li wrote:
>
> 在 2021/6/8 1:58, Nithin Dabilpuram 写道:
> > Add build infrastructure and common probe and remove for cnxk driver
> > which is used by both CN10K and CN9K SoC.
> >
> > Signed-off-by: Nithin Dabilpuram <ndabilpuram at marvell.com>
> > ---
> > MAINTAINERS | 3 +
> > doc/guides/nics/cnxk.rst | 29 +++++
> > doc/guides/nics/features/cnxk.ini | 9 ++
> > doc/guides/nics/features/cnxk_vec.ini | 9 ++
> > doc/guides/nics/features/cnxk_vf.ini | 9 ++
> > doc/guides/nics/index.rst | 1 +
> > doc/guides/platform/cnxk.rst | 3 +
> > drivers/net/cnxk/cnxk_ethdev.c | 219 ++++++++++++++++++++++++++++++++++
> > drivers/net/cnxk/cnxk_ethdev.h | 57 +++++++++
> > drivers/net/cnxk/meson.build | 21 ++++
> > drivers/net/cnxk/version.map | 3 +
> > drivers/net/meson.build | 1 +
> > 12 files changed, 364 insertions(+)
> > create mode 100644 doc/guides/nics/cnxk.rst
> > create mode 100644 doc/guides/nics/features/cnxk.ini
> > create mode 100644 doc/guides/nics/features/cnxk_vec.ini
> > create mode 100644 doc/guides/nics/features/cnxk_vf.ini
> > create mode 100644 drivers/net/cnxk/cnxk_ethdev.c
> > create mode 100644 drivers/net/cnxk/cnxk_ethdev.h
> > create mode 100644 drivers/net/cnxk/meson.build
> > create mode 100644 drivers/net/cnxk/version.map
> >
> > diff --git a/MAINTAINERS b/MAINTAINERS
> > index 5877a16..2be220e 100644
> > --- a/MAINTAINERS
> > +++ b/MAINTAINERS
> > @@ -746,6 +746,9 @@ M: Sunil Kumar Kori <skori at marvell.com>
> > M: Satha Rao <skoteshwar at marvell.com>
> > T: git://dpdk.org/next/dpdk-next-net-mrvl
> > F: drivers/common/cnxk/
> > +F: drivers/net/cnxk/
> > +F: doc/guides/nics/cnxk.rst
> > +F: doc/guides/nics/features/cnxk*.ini
> > F: doc/guides/platform/cnxk.rst
> > Marvell mvpp2
> > diff --git a/doc/guides/nics/cnxk.rst b/doc/guides/nics/cnxk.rst
> > new file mode 100644
> > index 0000000..ca21842
> > --- /dev/null
> > +++ b/doc/guides/nics/cnxk.rst
> > @@ -0,0 +1,29 @@
> > +.. SPDX-License-Identifier: BSD-3-Clause
> > + Copyright(C) 2021 Marvell.
> > +
> > +CNXK Poll Mode driver
> > +=====================
> > +
> > +The CNXK ETHDEV PMD (**librte_net_cnxk**) provides poll mode ethdev driver
> > +support for the inbuilt network device found in **Marvell OCTEON CN9K/CN10K**
> > +SoC family as well as for their virtual functions (VF) in SR-IOV context.
> > +
> > +More information can be found at `Marvell Official Website
> > +<https://www.marvell.com/embedded-processors/infrastructure-processors>`_.
> > +
> > +Features
> > +--------
> > +
> > +Features of the CNXK Ethdev PMD are:
> > +
> > +Prerequisites
> > +-------------
> > +
> > +See :doc:`../platform/cnxk` for setup information.
> > +
> > +
> > +Driver compilation and testing
> > +------------------------------
> > +
> > +Refer to the document :ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>`
> > +for details.
> > diff --git a/doc/guides/nics/features/cnxk.ini b/doc/guides/nics/features/cnxk.ini
> > new file mode 100644
> > index 0000000..2c23464
> > --- /dev/null
> > +++ b/doc/guides/nics/features/cnxk.ini
> > @@ -0,0 +1,9 @@
> > +;
> > +; Supported features of the 'cnxk' network poll mode driver.
> > +;
> > +; Refer to default.ini for the full list of available PMD features.
> > +;
> > +[Features]
> > +Linux = Y
> > +ARMv8 = Y
> > +Usage doc = Y
> > diff --git a/doc/guides/nics/features/cnxk_vec.ini b/doc/guides/nics/features/cnxk_vec.ini
> > new file mode 100644
> > index 0000000..de78516
> > --- /dev/null
> > +++ b/doc/guides/nics/features/cnxk_vec.ini
> > @@ -0,0 +1,9 @@
> > +;
> > +; Supported features of the 'cnxk_vec' network poll mode driver.
> > +;
> > +; Refer to default.ini for the full list of available PMD features.
> > +;
> > +[Features]
> > +Linux = Y
> > +ARMv8 = Y
> > +Usage doc = Y
> > diff --git a/doc/guides/nics/features/cnxk_vf.ini b/doc/guides/nics/features/cnxk_vf.ini
> > new file mode 100644
> > index 0000000..9c96351
> > --- /dev/null
> > +++ b/doc/guides/nics/features/cnxk_vf.ini
> > @@ -0,0 +1,9 @@
> > +;
> > +; Supported features of the 'cnxk_vf' network poll mode driver.
> > +;
> > +; Refer to default.ini for the full list of available PMD features.
> > +;
> > +[Features]
> > +Linux = Y
> > +ARMv8 = Y
> > +Usage doc = Y
> > diff --git a/doc/guides/nics/index.rst b/doc/guides/nics/index.rst
> > index 799697c..c1a04d9 100644
> > --- a/doc/guides/nics/index.rst
> > +++ b/doc/guides/nics/index.rst
> > @@ -19,6 +19,7 @@ Network Interface Controller Drivers
> > axgbe
> > bnx2x
> > bnxt
> > + cnxk
> > cxgbe
> > dpaa
> > dpaa2
> > diff --git a/doc/guides/platform/cnxk.rst b/doc/guides/platform/cnxk.rst
> > index cebb3d0..b506c11 100644
> > --- a/doc/guides/platform/cnxk.rst
> > +++ b/doc/guides/platform/cnxk.rst
> > @@ -142,6 +142,9 @@ HW Offload Drivers
> > This section lists dataplane H/W block(s) available in cnxk SoC.
> > +#. **Ethdev Driver**
> > + See :doc:`../nics/cnxk` for NIX Ethdev driver information.
> > +
> > #. **Mempool Driver**
> > See :doc:`../mempool/cnxk` for NPA mempool driver information.
> > diff --git a/drivers/net/cnxk/cnxk_ethdev.c b/drivers/net/cnxk/cnxk_ethdev.c
> > new file mode 100644
> > index 0000000..6717410
> > --- /dev/null
> > +++ b/drivers/net/cnxk/cnxk_ethdev.c
> > @@ -0,0 +1,219 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(C) 2021 Marvell.
> > + */
> > +#include <cnxk_ethdev.h>
> > +
> > +/* CNXK platform independent eth dev ops */
> > +struct eth_dev_ops cnxk_eth_dev_ops;
> > +
> > +static int
> > +cnxk_eth_dev_init(struct rte_eth_dev *eth_dev)
> > +{
> > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev);
> > + struct roc_nix *nix = &dev->nix;
> > + struct rte_pci_device *pci_dev;
> > + int rc, max_entries;
> > +
> > + eth_dev->dev_ops = &cnxk_eth_dev_ops;
> > +
> > + /* For secondary processes, the primary has done all the work */
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> > + return 0;
> > +
> > + pci_dev = RTE_ETH_DEV_TO_PCI(eth_dev);
> > + rte_eth_copy_pci_info(eth_dev, pci_dev);
> > + eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
> Hi,
>
> It is recommended not to use this flag when add new driver. Queue stats will
> be moved to xstats and be filled by PMDs.
>
> Please check the following patch:
>
> commit f30e69b41f949cd4a9afb6ff39de196e661708e2
> Author: Ferruh Yigit <ferruh.yigit at intel.com>
> Date: Wed Oct 14 03:26:47 2020 +0100
>
> ethdev: add device flag to bypass auto-filled queue xstats
>
> Queue stats are stored in 'struct rte_eth_stats' as array and array size
> is defined by 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag.
>
> As a result of technical board discussion, decided to remove the queue
> statistics from 'struct rte_eth_stats' in the long term.
>
> Instead PMDs should represent the queue statistics via xstats, this
> gives more flexibility on the number of the queues supported.
>
> Currently queue stats in the xstats are filled by ethdev layer, using
> some basic stats, when queue stats removed from basic stats the
> responsibility to fill the relevant xstats will be pushed to the PMDs.
>
> During the switch period, temporary 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS'
> device flag is created. Initially all PMDs using xstats set this flag.
> The PMDs implemented queue stats in the xstats should clear the flag.
>
> When all PMDs switch to the xstats for the queue stats, queue stats
> related fields from 'struct rte_eth_stats' will be removed, as well as
> 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' flag.
> Later 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag also can be
> removed.
>
> Signed-off-by: Ferruh Yigit <ferruh.yigit at intel.com>
> Acked-by: Haiyue Wang <haiyue.wang at intel.com>
> Acked-by: Xiao Wang <xiao.w.wang at intel.com>
> Acked-by: Thomas Monjalon <thomas at monjalon.net>
Ack, will fix it in V3.
>
> > +
> > + /* Initialize base roc nix */
> > + nix->pci_dev = pci_dev;
> > + rc = roc_nix_dev_init(nix);
> > + if (rc) {
> > + plt_err("Failed to initialize roc nix rc=%d", rc);
> > + goto error;
> > + }
> > +
> > + dev->eth_dev = eth_dev;
> > +
> > + /* For vfs, returned max_entries will be 0. but to keep default mac
> > + * address, one entry must be allocated. so setting up to 1.
> > + */
> > + if (roc_nix_is_vf_or_sdp(nix))
> > + max_entries = 1;
> > + else
> > + max_entries = roc_nix_mac_max_entries_get(nix);
> > +
> > + if (max_entries <= 0) {
> > + plt_err("Failed to get max entries for mac addr");
> > + rc = -ENOTSUP;
> > + goto dev_fini;
> > + }
> > +
> > + eth_dev->data->mac_addrs =
> > + rte_zmalloc("mac_addr", max_entries * RTE_ETHER_ADDR_LEN, 0);
> > + if (eth_dev->data->mac_addrs == NULL) {
> > + plt_err("Failed to allocate memory for mac addr");
> > + rc = -ENOMEM;
> > + goto dev_fini;
> > + }
> > +
> > + dev->max_mac_entries = max_entries;
> > +
> > + /* Get mac address */
> > + rc = roc_nix_npc_mac_addr_get(nix, dev->mac_addr);
> > + if (rc) {
> > + plt_err("Failed to get mac addr, rc=%d", rc);
> > + goto free_mac_addrs;
> > + }
> > +
> > + /* Update the mac address */
> > + memcpy(eth_dev->data->mac_addrs, dev->mac_addr, RTE_ETHER_ADDR_LEN);
> > +
> > + if (!roc_nix_is_vf_or_sdp(nix)) {
> > + /* Sync same MAC address to CGX/RPM table */
> > + rc = roc_nix_mac_addr_set(nix, dev->mac_addr);
> > + if (rc) {
> > + plt_err("Failed to set mac addr, rc=%d", rc);
> > + goto free_mac_addrs;
> > + }
> > + }
> > +
> > + /* Initialize roc npc */
> > + plt_nix_dbg("Port=%d pf=%d vf=%d ver=%s hwcap=0x%" PRIx64
> > + " rxoffload_capa=0x%" PRIx64 " txoffload_capa=0x%" PRIx64,
> > + eth_dev->data->port_id, roc_nix_get_pf(nix),
> > + roc_nix_get_vf(nix), CNXK_ETH_DEV_PMD_VERSION, dev->hwcap,
> > + dev->rx_offload_capa, dev->tx_offload_capa);
> > + return 0;
> > +
> > +free_mac_addrs:
> > + rte_free(eth_dev->data->mac_addrs);
> > +dev_fini:
> > + roc_nix_dev_fini(nix);
> > +error:
> > + plt_err("Failed to init nix eth_dev rc=%d", rc);
> > + return rc;
> > +}
> > +
> > +static int
> > +cnxk_eth_dev_uninit(struct rte_eth_dev *eth_dev, bool mbox_close)
> > +{
> > + struct cnxk_eth_dev *dev = cnxk_eth_pmd_priv(eth_dev);
> > + const struct eth_dev_ops *dev_ops = eth_dev->dev_ops;
> > + struct roc_nix *nix = &dev->nix;
> > + int rc, i;
> > +
> > + /* Nothing to be done for secondary processes */
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> > + return 0;
> > +
> > + roc_nix_npc_rx_ena_dis(nix, false);
> > +
> > + /* Free up SQs */
> > + for (i = 0; i < eth_dev->data->nb_tx_queues; i++) {
> > + dev_ops->tx_queue_release(eth_dev->data->tx_queues[i]);
> > + eth_dev->data->tx_queues[i] = NULL;
> > + }
> > + eth_dev->data->nb_tx_queues = 0;
> > +
> > + /* Free up RQ's and CQ's */
> > + for (i = 0; i < eth_dev->data->nb_rx_queues; i++) {
> > + dev_ops->rx_queue_release(eth_dev->data->rx_queues[i]);
> > + eth_dev->data->rx_queues[i] = NULL;
> > + }
> > + eth_dev->data->nb_rx_queues = 0;
> > +
> > + /* Free tm resources */
> > + roc_nix_tm_fini(nix);
> > +
> > + /* Unregister queue irqs */
> > + roc_nix_unregister_queue_irqs(nix);
> > +
> > + /* Unregister cq irqs */
> > + if (eth_dev->data->dev_conf.intr_conf.rxq)
> > + roc_nix_unregister_cq_irqs(nix);
> > +
> > + /* Free nix lf resources */
> > + rc = roc_nix_lf_free(nix);
> > + if (rc)
> > + plt_err("Failed to free nix lf, rc=%d", rc);
> > +
> > + rte_free(eth_dev->data->mac_addrs);
> > + eth_dev->data->mac_addrs = NULL;
> > +
> > + /* Check if mbox close is needed */
> > + if (!mbox_close)
> > + return 0;
> > +
> > + rc = roc_nix_dev_fini(nix);
> > + /* Can be freed later by PMD if NPA LF is in use */
> > + if (rc == -EAGAIN) {
> > + eth_dev->data->dev_private = NULL;
> > + return 0;
> > + } else if (rc) {
> > + plt_err("Failed in nix dev fini, rc=%d", rc);
> > + }
> > +
> > + return rc;
> > +}
> > +
> > +int
> > +cnxk_nix_remove(struct rte_pci_device *pci_dev)
> > +{
> > + struct rte_eth_dev *eth_dev;
> > + struct roc_nix *nix;
> > + int rc = -EINVAL;
> > +
> > + eth_dev = rte_eth_dev_allocated(pci_dev->device.name);
> > + if (eth_dev) {
> > + /* Cleanup eth dev */
> > + rc = cnxk_eth_dev_uninit(eth_dev, true);
> > + if (rc)
> > + return rc;
> > +
> > + rte_eth_dev_release_port(eth_dev);
> > + }
> > +
> > + /* Nothing to be done for secondary processes */
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY)
> > + return 0;
> > +
> > + /* Check if this device is hosting common resource */
> > + nix = roc_idev_npa_nix_get();
> > + if (nix->pci_dev != pci_dev)
> > + return 0;
> > +
> > + /* Try nix fini now */
> > + rc = roc_nix_dev_fini(nix);
> > + if (rc == -EAGAIN) {
> > + plt_info("%s: common resource in use by other devices",
> > + pci_dev->name);
> > + goto exit;
> > + } else if (rc) {
> > + plt_err("Failed in nix dev fini, rc=%d", rc);
> > + goto exit;
> > + }
> > +
> > + /* Free device pointer as rte_ethdev does not have it anymore */
> > + rte_free(nix);
> > +exit:
> > + return rc;
> > +}
> > +
> > +int
> > +cnxk_nix_probe(struct rte_pci_driver *pci_drv, struct rte_pci_device *pci_dev)
> > +{
> > + int rc;
> > +
> > + RTE_SET_USED(pci_drv);
> > +
> > + rc = rte_eth_dev_pci_generic_probe(pci_dev, sizeof(struct cnxk_eth_dev),
> > + cnxk_eth_dev_init);
> > +
> > + /* On error on secondary, recheck if port exists in primary or
> > + * in mid of detach state.
> > + */
> > + if (rte_eal_process_type() != RTE_PROC_PRIMARY && rc)
> > + if (!rte_eth_dev_allocated(pci_dev->device.name))
> > + return 0;
> > + return rc;
> > +}
> > diff --git a/drivers/net/cnxk/cnxk_ethdev.h b/drivers/net/cnxk/cnxk_ethdev.h
> > new file mode 100644
> > index 0000000..0460d1e
> > --- /dev/null
> > +++ b/drivers/net/cnxk/cnxk_ethdev.h
> > @@ -0,0 +1,57 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(C) 2021 Marvell.
> > + */
> > +#ifndef __CNXK_ETHDEV_H__
> > +#define __CNXK_ETHDEV_H__
> > +
> > +#include <math.h>
> > +#include <stdint.h>
> > +
> > +#include <ethdev_driver.h>
> > +#include <ethdev_pci.h>
> > +
> > +#include "roc_api.h"
> > +
> > +#define CNXK_ETH_DEV_PMD_VERSION "1.0"
> > +
> > +struct cnxk_eth_dev {
> > + /* ROC NIX */
> > + struct roc_nix nix;
> > +
> > + /* Max macfilter entries */
> > + uint8_t max_mac_entries;
> > +
> > + uint16_t flags;
> > +
> > + /* Pointer back to rte */
> > + struct rte_eth_dev *eth_dev;
> > +
> > + /* HW capabilities / Limitations */
> > + union {
> > + uint64_t hwcap;
> > + };
> > +
> > + /* Rx and Tx offload capabilities */
> > + uint64_t rx_offload_capa;
> > + uint64_t tx_offload_capa;
> > + uint32_t speed_capa;
> > +
> > + /* Default mac address */
> > + uint8_t mac_addr[RTE_ETHER_ADDR_LEN];
> > +};
> > +
> > +static inline struct cnxk_eth_dev *
> > +cnxk_eth_pmd_priv(struct rte_eth_dev *eth_dev)
> > +{
> > + return eth_dev->data->dev_private;
> > +}
> > +
> > +/* Common ethdev ops */
> > +extern struct eth_dev_ops cnxk_eth_dev_ops;
> > +
> > +/* Ops */
> > +int cnxk_nix_probe(struct rte_pci_driver *pci_drv,
> > + struct rte_pci_device *pci_dev);
> > +int cnxk_nix_remove(struct rte_pci_device *pci_dev);
> > +
> > +#endif /* __CNXK_ETHDEV_H__ */
> > diff --git a/drivers/net/cnxk/meson.build b/drivers/net/cnxk/meson.build
> > new file mode 100644
> > index 0000000..77b2f18
> > --- /dev/null
> > +++ b/drivers/net/cnxk/meson.build
> > @@ -0,0 +1,21 @@
> > +# SPDX-License-Identifier: BSD-3-Clause
> > +# Copyright(C) 2021 Marvell.
> > +#
> > +
> > +if not dpdk_conf.get('RTE_ARCH_64')
> > + build = false
> > + reason = 'only supported on 64-bit'
> > + subdir_done()
> > +endif
> > +
> > +sources = files('cnxk_ethdev.c')
> > +
> > +deps += ['bus_pci', 'cryptodev', 'eventdev', 'security']
> > +deps += ['common_cnxk', 'mempool_cnxk']
> > +
> > +extra_flags = ['-flax-vector-conversions', '-Wno-strict-aliasing']
> > +foreach flag: extra_flags
> > + if cc.has_argument(flag)
> > + cflags += flag
> > + endif
> > +endforeach
> > diff --git a/drivers/net/cnxk/version.map b/drivers/net/cnxk/version.map
> > new file mode 100644
> > index 0000000..ee80c51
> > --- /dev/null
> > +++ b/drivers/net/cnxk/version.map
> > @@ -0,0 +1,3 @@
> > +INTERNAL {
> > + local: *;
> > +};
> > diff --git a/drivers/net/meson.build b/drivers/net/meson.build
> > index c8b5ce2..5b066fd 100644
> > --- a/drivers/net/meson.build
> > +++ b/drivers/net/meson.build
> > @@ -12,6 +12,7 @@ drivers = [
> > 'bnx2x',
> > 'bnxt',
> > 'bonding',
> > + 'cnxk',
> > 'cxgbe',
> > 'dpaa',
> > 'dpaa2',
More information about the dev
mailing list