[dpdk-users] Issue with cryptodev_qat_autotest on CentOS 7.2 host system using dpdk-2.2.0

Chinmaya Dwibedy ckdwibedy at gmail.com
Wed Jun 22 15:57:11 CEST 2016
Previous message: [dpdk-users] Get interface speed through IOCTL
Next message: [dpdk-users] Issue with cryptodev_qat_autotest on CentOS 7.2 host system using dpdk-2.2.0
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi All,


I do have CentOS Linux release 7.2.1511 (Core) host system, where two Intel
QAT devices are installed. I have installed dpdk-2.2.0 and QAT driver as
mentioned in section 6.4. (i.e., Installation using 01.org QAT driver) of
http://dpdk.readthedocs.io/en/v16.04/cryptodevs/qat.html. I tested
"cryptodev_qat_autotest" under app/test. I found this test case to be
failed. It appears that, it the dequeue burst API fails to retrieve any
processed operations available from the queue pair (on the QAT Crypto
device), from the devices processed queue.  I have captured and attached
the console.log here.


Can anyone please suggest, is it an issue with the setup/configuration or
QAT PMD code? Kindly point me to right direction whenever you find me
wrong.Thank
you in advance for your time and support.


Here goes my configuration.



[root at localhost ~]# service qat_service status

There is 2 acceleration device(s) in the system:

 icp_dev0 - type=dh895xcc, inst_id=0, node_id=1,  bdf=83:00:0, #accel=6,
#engines=12, state=up

 icp_dev1 - type=dh895xcc, inst_id=1, node_id=1,  bdf=88:00:0, #accel=6,
#engines=12, state=up


[root at localhost ~]# lspci -nn | grep 0435

83:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT
[8086:0435]

88:00.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT
[8086:0435]

[root at localhost ~]#





[root at localhost ~]# lspci -nn|grep -e "QAT.*Virtual"

83:01.0 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

83:01.1 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

83:01.2 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

83:01.3 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

83:01.4 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

83:01.5 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

-------------------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------------------


88:04.1 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

88:04.2 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

88:04.3 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

88:04.4 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

88:04.5 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

88:04.6 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]

88:04.7 Co-processor [0b40]: Intel Corporation DH895XCC Series QAT Virtual
Function [8086:0443]


[root at localhost ~]# lspci -nn|grep -e "QAT.*Virtual" | wc -l

64

[root at localhost ~]#


Note: This host is  capable of PCI passthrough and SR-IOV.



[root at localhost ~]# grep -i dmar /var/log/dmesg

[    0.000000] ACPI: DMAR 000000007b69a000 00130 (v01 INTEL   S2600WT
00000001 INTL 20091013)

[    0.135336] dmar: Host address width 46

[    0.135587] dmar: DRHD base: 0x000000fbffc000 flags: 0x0

[    0.135855] dmar: IOMMU 0: reg_base_addr fbffc000 ver 1:0 cap
d2078c106f0466 ecap f020de

[    0.136309] dmar: DRHD base: 0x000000c7ffc000 flags: 0x1

[    0.136569] dmar: IOMMU 1: reg_base_addr c7ffc000 ver 1:0 cap
d2078c106f0466 ecap f020de

[    0.137016] dmar: RMRR base: 0x0000007a3db000 end: 0x0000007a3ddfff

[    0.137274] dmar: ATSR flags: 0x0

[    0.137522] dmar: ATSR flags: 0x0

[root at localhost ~]# grep -i "iommu.*enabled" /var/log/dmesg

[    0.000000] Intel-IOMMU: enabled


[root at localhost test(keystone_admin)]# cat /proc/cmdline

BOOT_IMAGE=/vmlinuz-3.10.0-327.18.2.el7.x86_64 root=/dev/mapper/centos-root
ro rd.lvm.lv=centos/swap crashkernel=auto nomodeset rd.lvm.lv=centos/root
rhgb iommu=pt intel_iommu=on

[root at localhost test(keystone_admin)]#


The below output confirms that, QAT driver is correctly installed.


 [root at localhost ~]# cat /proc/icp_dh895xcc_dev0/version

+--------------------------------------------------+

| Hardware and Software versions for device 0      |

+--------------------------------------------------+

 Hardware Version:             A0 SKU2

 Firmware Version:             2.6.0

 MMP Version:                  1.0.0

 Driver Version:               2.6.0

 Lowest Compatible Driver:     2.3

 QuickAssist API CY Version:   1.9

 QuickAssist API DC Version:   1.6

+--------------------------------------------------+

[root at localhost ~]# cat /proc/icp_dh895xcc_dev1/version

+--------------------------------------------------+

| Hardware and Software versions for device 1      |

+--------------------------------------------------+

 Hardware Version:             A0 SKU2

 Firmware Version:             2.6.0

 MMP Version:                  1.0.0

 Driver Version:               2.6.0

 Lowest Compatible Driver:     2.3

 QuickAssist API CY Version:   1.9

 QuickAssist API DC Version:   1.6

+--------------------------------------------------+

[root at localhost ~]#



The # lspci -d:443  confirms the bdf of the 32 VF devices are available per
QAT device.



[root at localhost ~]# lspci -d:443

83:01.0 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

83:01.1 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

83:01.2 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

83:01.3 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

-------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------

88:04.3 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

88:04.4 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

88:04.5 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

88:04.6 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

88:04.7 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

[root at localhost ~]#



*DPDK configuration options in config/defconfig_x86_64-native-linuxapp-gcc*



CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n

CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=y

CONFIG_RTE_LIBRTE_VIRTIO_PMD=n

CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_INIT=n

CONFIG_RTE_LIBRTE_VIRTIO_DEBUG_DRIVER=n

CONFIG_RTE_LIBRTE_PMD_QAT=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=y

CONFIG_RTE_MAX_MEMSEG=4096

[root at localhost dpdk-2.2.0]#



Here goes my setup.sh



make config T=x86_64-native-linuxapp-gcc

sed -ri 's,(PMD_PCAP=).*,\1y,' build/.config

make install T=x86_64-native-linuxapp-gcc

mkdir -p /mnt/huge

mount -t hugetlbfs nodev /mnt/huge

echo 8192 >
/sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages

echo 8192 >
/sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages

sudo modprobe uio_pci_generic

sudo modprobe uio

rmmod igb_uio

insmod
./x86_64-native-linuxapp-gcc/build/lib/librte_eal/linuxapp/igb_uio/igb_uio.ko

echo -n "0000:83:01.0" > /sys/bus/pci/devices/0000\:83\:01.0/driver/unbind

echo "8086 0443" > /sys/bus/pci/drivers/igb_uio/new_id


The below OP confirms that, the QAT VF is bound to the DPDK UIO driver.



83:01.0 Co-processor: Intel Corporation DH895XCC Series QAT Virtual Function

        Subsystem: Intel Corporation Device 0000

        Control: I/O- Mem- BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-

        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-

        Latency: 0

        Region 0: [virtual] Memory at c8280000 (64-bit, non-prefetchable)
[size=4K]

        Region 2: [virtual] Memory at c82a0000 (64-bit, non-prefetchable)
[size=4K]

        Capabilities: [90] MSI: Enable- Count=1/1 Maskable+ 64bit+

                Address: 0000000000000000  Data: 0000

                Masking: 00000000  Pending: 00000000

        Kernel driver in use: igb_uio


Regards,

Chinmaya
Previous message: [dpdk-users] Get interface speed through IOCTL
Next message: [dpdk-users] Issue with cryptodev_qat_autotest on CentOS 7.2 host system using dpdk-2.2.0
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the users mailing list