[dpdk-dev] [PATCH v3 2/3] doc: restructure QAT PMD guide
Fiona Trahe
fiona.trahe at intel.com
Mon Apr 3 16:13:22 CEST 2017
Restructure QAT PMD instructions and add a device table to
minimise duplication for each device and make it easier to add devices.
Fix some device name typos and poor formatting.
Signed-off-by: Fiona Trahe <fiona.trahe at intel.com>
---
doc/guides/cryptodevs/qat.rst | 416 ++++++++++++++++++------------------------
1 file changed, 179 insertions(+), 237 deletions(-)
diff --git a/doc/guides/cryptodevs/qat.rst b/doc/guides/cryptodevs/qat.rst
index 9ecd19b..d6f1773 100644
--- a/doc/guides/cryptodevs/qat.rst
+++ b/doc/guides/cryptodevs/qat.rst
@@ -30,9 +30,12 @@
Intel(R) QuickAssist (QAT) Crypto Poll Mode Driver
==================================================
-The QAT PMD provides poll mode crypto driver support for **Intel QuickAssist
-Technology DH895xxC**, **Intel QuickAssist Technology C62x** and
-**Intel QuickAssist Technology C3xxx** hardware accelerator.
+The QAT PMD provides poll mode crypto driver support for the following
+hardware accelerator devices:
+
+* ``Intel QuickAssist Technology DH895xCC``
+* ``Intel QuickAssist Technology C62x``
+* ``Intel QuickAssist Technology C3xxx``
Features
@@ -84,332 +87,271 @@ Limitations
Installation
------------
-To use the DPDK QAT PMD an SRIOV-enabled QAT kernel driver is required. The
-VF devices exposed by this driver will be used by QAT PMD.
-
-To enable QAT in DPDK, follow the instructions mentioned in
-http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html
+To enable QAT in DPDK, follow the instructions for modifying the compile-time
+configuration file as described `here <http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html>`_.
-Quick instructions as follows:
+Quick instructions are as follows:
.. code-block:: console
+ cd to the top-level DPDK directory
make config T=x86_64-native-linuxapp-gcc
sed -i 's,\(CONFIG_RTE_LIBRTE_PMD_QAT\)=n,\1=y,' build/.config
make
-If you are running on kernel 4.4 or greater, see instructions for
-`Installation using kernel.org driver`_ below. If you are on a kernel earlier
-than 4.4, see `Installation using 01.org QAT driver`_.
-
-For **Intel QuickAssist Technology C62x** and **Intel QuickAssist Technology C3xxx**
-device, kernel 4.5 or greater is needed.
-See instructions for `Installation using kernel.org driver`_ below.
-
-
-Installation using 01.org QAT driver
-------------------------------------
-
-NOTE: There is no driver available for **Intel QuickAssist Technology C62x** and
-**Intel QuickAssist Technology C3xxx** devices on 01.org.
-
-Download the latest QuickAssist Technology Driver from `01.org
-<https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches>`_
-Consult the *Getting Started Guide* at the same URL for further information.
+To use the DPDK QAT PMD an SRIOV-enabled QAT kernel driver is required. The VF
+devices exposed by this driver will be used by the QAT PMD. The devices and
+available kernel drivers and device ids are :
-The steps below assume you are:
+.. _table_qat_pmds_drivers:
-* Building on a platform with one ``DH895xCC`` device.
-* Using package ``qatmux.l.2.3.0-34.tgz``.
-* On Fedora21 kernel ``3.17.4-301.fc21.x86_64``.
+.. table:: QAT devices and drivers
-In the BIOS ensure that SRIOV is enabled and VT-d is disabled.
+ +----------+--------+---------------+------------+--------+---------+--------+------------+
+ | Device | Driver | Kernel Module | Pci Driver | PF Did | Num PFs | Vf Did | VFs per PF |
+ +==========+========+===============+============+========+=========+========+============+
+ | DH895xCC | 01.org | icp_qa_al | n/a | 435 | 1 | 443 | 32 |
+ +----------+--------+---------------+------------+--------+---------+--------+------------+
+ | DH895xCC | 4.4+ | qat_dh895xcc | dh895xcc | 435 | 1 | 443 | 32 |
+ +----------+--------+---------------+------------+--------+---------+--------+------------+
+ | C62x | 4.5+ | qat_c62x | c6xx | 37c8 | 3 | 37c9 | 16 |
+ +----------+--------+---------------+------------+--------+---------+--------+------------+
+ | C3xxx | 4.5+ | qat_c3xxx | c3xxx | 19e2 | 1 | 19e3 | 16 |
+ +----------+--------+---------------+------------+--------+---------+--------+------------+
-Uninstall any existing QAT driver, for example by running:
-* ``./installer.sh uninstall`` in the directory where originally installed.
+The ``Driver`` column indicates either the Linux kernel version in which
+support for this device was introduced or a driver available on Intel's 01.org
+website. There are both linux and 01.org kernel drivers available for some
+devices.
-* or ``rmmod qat_dh895xcc; rmmod intel_qat``.
-
-Build and install the SRIOV-enabled QAT driver::
-
- mkdir /QAT
- cd /QAT
- # copy qatmux.l.2.3.0-34.tgz to this location
- tar zxof qatmux.l.2.3.0-34.tgz
-
- export ICP_WITHOUT_IOMMU=1
- ./installer.sh install QAT1.6 host
-
-You can use ``cat /proc/icp_dh895xcc_dev0/version`` to confirm the driver is correctly installed.
-You can use ``lspci -d:443`` to confirm the bdf of the 32 VF devices are available per ``DH895xCC`` device.
-
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
-
-**Note**: If using a later kernel and the build fails with an error relating to ``strict_stroul`` not being available apply the following patch:
-
-.. code-block:: diff
-
- /QAT/QAT1.6/quickassist/utilities/downloader/Target_CoreLibs/uclo/include/linux/uclo_platform.h
- + #if LINUX_VERSION_CODE >= KERNEL_VERSION(3,18,5)
- + #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (kstrtoul((str), (base), (num))) printk("Error strtoull convert %s\n", str); }
- + #else
- #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,38)
- #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (strict_strtoull((str), (base), (num))) printk("Error strtoull convert %s\n", str); }
- #else
- #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,25)
- #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; strict_strtoll((str), (base), (num));}
- #else
- #define STR_TO_64(str, base, num, endPtr) \
- do { \
- if (str[0] == '-') \
- { \
- *(num) = -(simple_strtoull((str+1), &(endPtr), (base))); \
- }else { \
- *(num) = simple_strtoull((str), &(endPtr), (base)); \
- } \
- } while(0)
- + #endif
- #endif
- #endif
-
-
-If the build fails due to missing header files you may need to do following:
-
-* ``sudo yum install zlib-devel``
-* ``sudo yum install openssl-devel``
-
-If the build or install fails due to mismatching kernel sources you may need to do the following:
-
-* ``sudo yum install kernel-headers-`uname -r```
-* ``sudo yum install kernel-src-`uname -r```
-* ``sudo yum install kernel-devel-`uname -r```
+If you are running on a kernel which includes a driver for your device, see
+`Installation using kernel.org driver`_ below. Otherwise see
+`Installation using 01.org QAT driver`_.
Installation using kernel.org driver
------------------------------------
-For **Intel QuickAssist Technology DH895xxC**:
-
-Assuming you are running on at least a 4.4 kernel, you can use the stock kernel.org QAT
-driver to start the QAT hardware.
-
-The steps below assume you are:
+The examples below are based on the C62x device, if you have a different device
+use the corresponding values in the above table.
-* Running DPDK on a platform with one ``DH895xCC`` device.
-* On a kernel at least version 4.4.
+In BIOS ensure that SRIOV is enabled and either:
-In BIOS ensure that SRIOV is enabled and either
-a) disable VT-d or
-b) enable VT-d and set ``"intel_iommu=on iommu=pt"`` in the grub file.
+* Disable VT-d or
+* Enable VT-d and set ``"intel_iommu=on iommu=pt"`` in the grub file.
-Ensure the QAT driver is loaded on your system, by executing::
+Check that the QAT driver is loaded on your system, by executing::
- lsmod | grep qat
+ lsmod | grep qa
-You should see the following output::
+You should see the kernel module for your device listed, e.g.::
- qat_dh895xcc 5626 0
- intel_qat 82336 1 qat_dh895xcc
+ qat_c62x 5626 0
+ intel_qat 82336 1 qat_c62x
Next, you need to expose the Virtual Functions (VFs) using the sysfs file system.
-First find the bdf of the physical function (PF) of the DH895xCC device::
+First find the BDFs (Bus-Device-Function) of the physical functions (PFs) of
+your device, e.g.::
- lspci -d : 435
+ lspci -d : 37c8
You should see output similar to::
- 03:00.0 Co-processor: Intel Corporation Coleto Creek PCIe Endpoint
-
-Using the sysfs, enable the VFs::
-
- echo 32 > /sys/bus/pci/drivers/dh895xcc/0000\:03\:00.0/sriov_numvfs
-
-If you get an error, it's likely you're using a QAT kernel driver earlier than kernel 4.4.
-
-To verify that the VFs are available for use - use ``lspci -d:443`` to confirm
-the bdf of the 32 VF devices are available per ``DH895xCC`` device.
+ 1a:00.0 Co-processor: Intel Corporation Device 37c8
+ 3d:00.0 Co-processor: Intel Corporation Device 37c8
+ 3f:00.0 Co-processor: Intel Corporation Device 37c8
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
+Enable the VFs for each PF by echoing the number of VFs per PF to the pci driver::
-**Note**: If the QAT kernel modules are not loaded and you see an error like
- ``Failed to load MMP firmware qat_895xcc_mmp.bin`` this may be as a
- result of not using a distribution, but just updating the kernel directly.
+ echo 16 > /sys/bus/pci/drivers/c6xx/0000:1a:00.0/sriov_numvfs
+ echo 16 > /sys/bus/pci/drivers/c6xx/0000:3d:00.0/sriov_numvfs
+ echo 16 > /sys/bus/pci/drivers/c6xx/0000:3f:00.0/sriov_numvfs
-Download firmware from the kernel firmware repo at:
-http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/
+Check that the VFs are available for use. For example ``lspci -d:37c9`` should
+list 48 VF devices available for a ``C62x`` device.
-Copy qat binaries to /lib/firmware:
-* ``cp qat_895xcc.bin /lib/firmware``
-* ``cp qat_895xcc_mmp.bin /lib/firmware``
+To complete the installation follow the instructions in
+`Binding the available VFs to the DPDK UIO driver`_.
-cd to your linux source root directory and start the qat kernel modules:
-* ``insmod ./drivers/crypto/qat/qat_common/intel_qat.ko``
-* ``insmod ./drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko``
+.. Note::
-**Note**:The following warning in /var/log/messages can be ignored:
- ``IOMMU should be enabled for SR-IOV to work correctly``
+ If the QAT kernel modules are not loaded and you see an error like ``Failed
+ to load MMP firmware qat_895xcc_mmp.bin`` in kernel logs, this may be as a
+ result of not using a distribution, but just updating the kernel directly.
-For **Intel QuickAssist Technology C62x**:
-Assuming you are running on at least a 4.5 kernel, you can use the stock kernel.org QAT
-driver to start the QAT hardware.
+ Download firmware from the `kernel firmware repo
+ <http://git.kernel.org/cgit/linux/kernel/git/firmware/linux-firmware.git/tree/>`_.
-The steps below assume you are:
+ Copy qat binaries to ``/lib/firmware``::
-* Running DPDK on a platform with one ``C62x`` device.
-* On a kernel at least version 4.5.
+ cp qat_895xcc.bin /lib/firmware
+ cp qat_895xcc_mmp.bin /lib/firmware
-In BIOS ensure that SRIOV is enabled and either
-a) disable VT-d or
-b) enable VT-d and set ``"intel_iommu=on iommu=pt"`` in the grub file.
+ Change to your linux source root directory and start the qat kernel modules::
-Ensure the QAT driver is loaded on your system, by executing::
+ insmod ./drivers/crypto/qat/qat_common/intel_qat.ko
+ insmod ./drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko
- lsmod | grep qat
-You should see the following output::
+.. Note::
- qat_c62x 16384 0
- intel_qat 122880 1 qat_c62x
+ If you see the following warning in ``/var/log/messages`` it can be ignored:
+ ``IOMMU should be enabled for SR-IOV to work correctly``.
-Next, you need to expose the VFs using the sysfs file system.
-First find the bdf of the C62x device::
+Installation using 01.org QAT driver
+------------------------------------
- lspci -d:37c8
+Download the latest QuickAssist Technology Driver from `01.org
+<https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches>`_.
+Consult the *Getting Started Guide* at the same URL for further information.
-You should see output similar to::
+The steps below assume you are:
- 1a:00.0 Co-processor: Intel Corporation Device 37c8
- 3d:00.0 Co-processor: Intel Corporation Device 37c8
- 3f:00.0 Co-processor: Intel Corporation Device 37c8
+* Building on a platform with one ``DH895xCC`` device.
+* Using package ``qatmux.l.2.3.0-34.tgz``.
+* On Fedora21 kernel ``3.17.4-301.fc21.x86_64``.
-For each c62x device there are 3 PFs.
-Using the sysfs, for each PF, enable the 16 VFs::
+In the BIOS ensure that SRIOV is enabled and VT-d is disabled.
- echo 16 > /sys/bus/pci/drivers/c6xx/0000\:1a\:00.0/sriov_numvfs
+Uninstall any existing QAT driver, for example by running:
-If you get an error, it's likely you're using a QAT kernel driver earlier than kernel 4.5.
+* ``./installer.sh uninstall`` in the directory where originally installed.
-To verify that the VFs are available for use - use ``lspci -d:37c9`` to confirm
-the bdf of the 48 VF devices are available per ``C62x`` device.
+* or ``rmmod qat_dh895xcc; rmmod intel_qat``.
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
+Build and install the SRIOV-enabled QAT driver::
-For **Intel QuickAssist Technology C3xxx**:
-Assuming you are running on at least a 4.5 kernel, you can use the stock kernel.org QAT
-driver to start the QAT hardware.
+ mkdir /QAT
+ cd /QAT
-The steps below assume you are:
+ # Copy qatmux.l.2.3.0-34.tgz to this location
+ tar zxof qatmux.l.2.3.0-34.tgz
-* Running DPDK on a platform with one ``C3xxx`` device.
-* On a kernel at least version 4.5.
+ export ICP_WITHOUT_IOMMU=1
+ ./installer.sh install QAT1.6 host
-In BIOS ensure that SRIOV is enabled and either
-a) disable VT-d or
-b) enable VT-d and set ``"intel_iommu=on iommu=pt"`` in the grub file.
+You can use ``cat /proc/icp_dh895xcc_dev0/version`` to confirm the driver is correctly installed.
+You can use ``lspci -d:443`` to confirm the of the 32 VF devices available per ``DH895xCC`` device.
-Ensure the QAT driver is loaded on your system, by executing::
+To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
- lsmod | grep qat
+.. Note::
-You should see the following output::
+ If using a later kernel and the build fails with an error relating to
+ ``strict_stroul`` not being available apply the following patch:
- qat_c3xxx 16384 0
- intel_qat 122880 1 qat_c3xxx
+ .. code-block:: diff
-Next, you need to expose the Virtual Functions (VFs) using the sysfs file system.
+ /QAT/QAT1.6/quickassist/utilities/downloader/Target_CoreLibs/uclo/include/linux/uclo_platform.h
+ + #if LINUX_VERSION_CODE >= KERNEL_VERSION(3,18,5)
+ + #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (kstrtoul((str), (base), (num))) printk("Error strtoull convert %s\n", str); }
+ + #else
+ #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,38)
+ #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; if (strict_strtoull((str), (base), (num))) printk("Error strtoull convert %s\n", str); }
+ #else
+ #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,25)
+ #define STR_TO_64(str, base, num, endPtr) {endPtr=NULL; strict_strtoll((str), (base), (num));}
+ #else
+ #define STR_TO_64(str, base, num, endPtr) \
+ do { \
+ if (str[0] == '-') \
+ { \
+ *(num) = -(simple_strtoull((str+1), &(endPtr), (base))); \
+ }else { \
+ *(num) = simple_strtoull((str), &(endPtr), (base)); \
+ } \
+ } while(0)
+ + #endif
+ #endif
+ #endif
-First find the bdf of the physical function (PF) of the C3xxx device
- lspci -d:19e2
+.. Note::
-You should see output similar to::
+ If the build fails due to missing header files you may need to do following::
- 01:00.0 Co-processor: Intel Corporation Device 19e2
+ sudo yum install zlib-devel
+ sudo yum install openssl-devel
-For c3xxx device there is 1 PFs.
-Using the sysfs, enable the 16 VFs::
+.. Note::
- echo 16 > /sys/bus/pci/drivers/c3xxx/0000\:01\:00.0/sriov_numvfs
+ If the build or install fails due to mismatching kernel sources you may need to do the following::
-If you get an error, it's likely you're using a QAT kernel driver earlier than kernel 4.5.
+ sudo yum install kernel-headers-`uname -r`
+ sudo yum install kernel-src-`uname -r`
+ sudo yum install kernel-devel-`uname -r`
-To verify that the VFs are available for use - use ``lspci -d:19e3`` to confirm
-the bdf of the 16 VF devices are available per ``C3xxx`` device.
-To complete the installation - follow instructions in `Binding the available VFs to the DPDK UIO driver`_.
Binding the available VFs to the DPDK UIO driver
------------------------------------------------
-For **Intel(R) QuickAssist Technology DH895xcc** device:
-The unbind command below assumes ``bdfs`` of ``03:01.00-03:04.07``, if yours are different adjust the unbind command below::
+Unbind the VFs from the stock driver so they can be bound to the uio driver.
- cd $RTE_SDK
- modprobe uio
- insmod ./build/kmod/igb_uio.ko
+For an Intel(R) QuickAssist Technology DH895xCC device
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- for device in $(seq 1 4); do \
- for fn in $(seq 0 7); do \
- echo -n 0000:03:0${device}.${fn} > \
- /sys/bus/pci/devices/0000\:03\:0${device}.${fn}/driver/unbind; \
- done; \
- done
+The unbind command below assumes ``BDFs`` of ``03:01.00-03:04.07``, if your
+VFs are different adjust the unbind command below::
- echo "8086 0443" > /sys/bus/pci/drivers/igb_uio/new_id
+ for device in $(seq 1 4); do \
+ for fn in $(seq 0 7); do \
+ echo -n 0000:03:0${device}.${fn} > \
+ /sys/bus/pci/devices/0000\:03\:0${device}.${fn}/driver/unbind; \
+ done; \
+ done
-You can use ``lspci -vvd:443`` to confirm that all devices are now in use by igb_uio kernel driver.
+For an Intel(R) QuickAssist Technology C62x device
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-For **Intel(R) QuickAssist Technology C62x** device:
-The unbind command below assumes ``bdfs`` of ``1a:01.00-1a:02.07``, ``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``,
-if yours are different adjust the unbind command below::
+The unbind command below assumes ``BDFs`` of ``1a:01.00-1a:02.07``,
+``3d:01.00-3d:02.07`` and ``3f:01.00-3f:02.07``, if your VFs are different
+adjust the unbind command below::
- cd $RTE_SDK
- modprobe uio
- insmod ./build/kmod/igb_uio.ko
+ for device in $(seq 1 2); do \
+ for fn in $(seq 0 7); do \
+ echo -n 0000:1a:0${device}.${fn} > \
+ /sys/bus/pci/devices/0000\:1a\:0${device}.${fn}/driver/unbind; \
- for device in $(seq 1 2); do \
- for fn in $(seq 0 7); do \
- echo -n 0000:1a:0${device}.${fn} > \
- /sys/bus/pci/devices/0000\:1a\:0${device}.${fn}/driver/unbind; \
+ echo -n 0000:3d:0${device}.${fn} > \
+ /sys/bus/pci/devices/0000\:3d\:0${device}.${fn}/driver/unbind; \
- echo -n 0000:3d:0${device}.${fn} > \
- /sys/bus/pci/devices/0000\:3d\:0${device}.${fn}/driver/unbind; \
+ echo -n 0000:3f:0${device}.${fn} > \
+ /sys/bus/pci/devices/0000\:3f\:0${device}.${fn}/driver/unbind; \
+ done; \
+ done
- echo -n 0000:3f:0${device}.${fn} > \
- /sys/bus/pci/devices/0000\:3f\:0${device}.${fn}/driver/unbind; \
- done; \
- done
+For Intel(R) QuickAssist Technology C3xxx device
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
+The unbind command below assumes ``BDFs`` of ``01:01.00-01:02.07``, if your
+VFs are different adjust the unbind command below::
-You can use ``lspci -vvd:37c9`` to confirm that all devices are now in use by igb_uio kernel driver.
+ for device in $(seq 1 2); do \
+ for fn in $(seq 0 7); do \
+ echo -n 0000:01:0${device}.${fn} > \
+ /sys/bus/pci/devices/0000\:01\:0${device}.${fn}/driver/unbind; \
+ done; \
+ done
-For **Intel(R) QuickAssist Technology C3xxx** device:
-The unbind command below assumes ``bdfs`` of ``01:01.00-01:02.07``,
-if yours are different adjust the unbind command below::
+Bind to the DPDK uio driver
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
- cd $RTE_SDK
- modprobe uio
- insmod ./build/kmod/igb_uio.ko
+Install the DPDK igb_uio driver, bind the VF PCI Device id to it and use lspci
+to confirm the VF devices are now in use by igb_uio kernel driver,
+e.g. for the C62x device::
- for device in $(seq 1 2); do \
- for fn in $(seq 0 7); do \
- echo -n 0000:01:0${device}.${fn} > \
- /sys/bus/pci/devices/0000\:01\:0${device}.${fn}/driver/unbind; \
+ cd to the top-level DPDK directory
+ modprobe uio
+ insmod ./build/kmod/igb_uio.ko
+ echo "8086 37c9" > /sys/bus/pci/drivers/igb_uio/new_id
+ lspci -vvd:37c9
- done; \
- done
- echo "8086 19e3" > /sys/bus/pci/drivers/igb_uio/new_id
-
-You can use ``lspci -vvd:19e3`` to confirm that all devices are now in use by igb_uio kernel driver.
-
-
-The other way to bind the VFs to the DPDK UIO driver is by using the ``dpdk-devbind.py`` script:
-
-.. code-block:: console
+Another way to bind the VFs to the DPDK UIO driver is by using the
+``dpdk-devbind.py`` script::
- cd $RTE_SDK
+ cd to the top-level DPDK directory
./usertools/dpdk-devbind.py -b igb_uio 0000:03:01.1
--
2.5.0
More information about the dev
mailing list