[dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)

Chinmaya Dwibedy ckdwibedy at gmail.com
Mon Jun 20 09:33:25 CEST 2016
Previous message: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)
Next message: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi All,


@Sergio:  Thank you for your valuable suggestion.


I passed through one of VFs and it detected the QAT device in VM. I just
sent one UDP datagram which has to be encapsulated using H/W crypto device
(i.e., Intel QAT). I run the ipsec-segw application with following
arguments.  ./build/ipsec-secgw  -l 0 -n 4  -- -p 0x3 -P
--config="(0,0,0),(1,0,0)" --cdev QAT --ep0. Found that, the
rte_crypto_enqueue_burst() function returns one. It means it could able to
suceesfully place packet on the queue “queue_id” of the QAT device
designated by its “dev_id”*.  But The rte_crypto_dequeue_burst() function
returns zero. It means it cannot dequeue the packet and that packet might
not have been processed by QAT device.


Please note that, I have tested the same application with AESNI device and
did not encounter any such issue. Furthermore the
rte_crypto_dequeue_burst() API does not provide any error notification.
Can anyone please suggest what might be the issue and is there any way to
debug further?

Here are the configuration options and console logs .



[root at vpn-s ipsec-secgw]# lspci -nn | grep 0443

00:06.0 Co-processor [0b40]: Intel Corporation Device [8086:0443]

[root at vpn-s ipsec-secgw]#


I have configured the followings in config file


CONFIG_RTE_LIBRTE_PMD_QAT=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_INIT=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_TX=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_RX=y

CONFIG_RTE_LIBRTE_PMD_QAT_DEBUG_DRIVER=y



Console log while running ipsec-segw sample application



PMD: crypto_qat_dev_init():  >>

PMD: crypto_qat_dev_init(): Found crypto device at 00:06.0

Promiscuous mode selected

endpoint 0

Found 1 QAT devices

PMD: qat_dev_info_get():  >>

Initialising QAT device 0

CRYPTODEV: rte_cryptodev_queue_pairs_config() line 467: Setup 1 queues
pairs on device 0

PMD: qat_dev_info_get():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

PMD: qat_crypto_sym_session_init():  >>

CRYPTODEV: rte_crypto_session_pool_create() line 1031: cdev_0_sess_mp
mempool created!

PMD: qat_crypto_sym_qp_setup():  >>

PMD: qat_tx_queue_create():  >>

PMD: qat_tx_queue_create(): TX ring for 2048 msgs: qp_id 0, bundle 0, ring 2

PMD: qat_queue_create():  >>

PMD: queue_dma_zone_reserve():  >>

PMD: queue_dma_zone_reserve(): Allocate memzone for
rte_qat_pmd_qp_mem_0_0_2, size 262144 on socket 0

PMD: qat_qp_check_queue_alignment():  >>

PMD: adf_verify_queue_size():  >>

PMD: qat_queue_create(): RING size in CSR: 12, in bytes 262144, nb msgs
2048, msg_size 128, max_inflights 2047 modulo 18

PMD: qat_rx_queue_create():  >>

PMD: qat_rx_queue_create(): RX ring for 2048 msgs: qp id 0, bundle 0, ring
10

PMD: qat_queue_create():  >>

PMD: queue_dma_zone_reserve():  >>

PMD: queue_dma_zone_reserve(): Allocate memzone for
rte_qat_pmd_qp_mem_0_0_10, size 65536 on socket 0

PMD: qat_qp_check_queue_alignment():  >>

PMD: adf_verify_queue_size():  >>

PMD: qat_queue_create(): RING size in CSR: 10, in bytes 65536, nb msgs
2048, msg_size 32, max_inflights 2047 modulo 16

PMD: adf_configure_queues():  >>

PMD: adf_queue_arb_enable():  >>

PMD: qat_dev_info_get():  >>

Creating SA context with 64 maximum entries

Creating SA context with 64 maximum entries

PMD: qat_crypto_sym_configure_session():  >>

PMD: qat_alg_aead_session_create_content_desc():  >>

PMD: qat_alg_do_precomputes():  >>

PMD: partial_hash_compute():  >>

PMD: partial_hash_compute():  >>

PMD: qat_alg_init_common_hdr():  >>

PMD: qat_crypto_sym_configure_session():  >>

PMD: qat_alg_aead_session_create_content_desc():  >>

PMD: qat_alg_do_precomputes():  >>

PMD: partial_hash_compute():  >>

PMD: partial_hash_compute():  >>

PMD: qat_alg_init_common_hdr():  >>

PMD: qat_crypto_sym_configure_session():  >>

PMD: qat_alg_aead_session_create_content_desc():  >>

PMD: partial_hash_compute():  >>

PMD: qat_alg_init_common_hdr():  >>

Creating SP context with 1000 max rules

IPv4 sp_ipv4_in_0 entries [4]:

        1:0.0.0.0/0 30.1.1.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x1-0x5

        2:0.0.0.0/0 192.168.116.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x2-0x6

        3:0.0.0.0/0 192.168.117.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x3-0x7

        4:0.0.0.0/0 192.168.118.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x4-0x8

ACL: Gen phase for ACL "sp_ipv4_in_0":

runtime memory footprint on socket 0:

single nodes/bytes used: 4/32

quad nodes/vectors/bytes used: 5/19/152

DFA nodes/group64/bytes used: 1/4/4104

match nodes/bytes used: 4/512

total: 7008 bytes

max limit: 18446744073709551615 bytes

ACL: Build phase for ACL "sp_ipv4_in_0":

node limit for tree split: 2048

nodes created: 14

memory consumed: 8388615

ACL: trie 0: number of rules: 4, indexes: 2

acl context <sp_ipv4_in_0>@0x7faefcb88500

  socket_id=0

  alg=3

  max_rules=1000

  rule_size=96

  num_rules=4

  num_categories=1

  num_tries=1

Creating SP context with 1000 max rules

IPv4 sp_ipv4_out_0 entries [4]:

        1:0.0.0.0/0 30.1.1.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x1-0x5

        2:0.0.0.0/0 192.168.4.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x2-0x6

        3:0.0.0.0/0 192.168.5.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x3-0x7

        4:0.0.0.0/0 192.168.6.0/24 0 : 65535 0 : 65535 0x0/0x0 0x1-0x4-0x8

ACL: Gen phase for ACL "sp_ipv4_out_0":

runtime memory footprint on socket 0:

single nodes/bytes used: 4/32

quad nodes/vectors/bytes used: 5/19/152

DFA nodes/group64/bytes used: 1/4/4104

match nodes/bytes used: 4/512

total: 7008 bytes

max limit: 18446744073709551615 bytes

ACL: Build phase for ACL "sp_ipv4_out_0":

node limit for tree split: 2048

-------------------------------------



Port 0 Link Up - speed 10000 Mbps - full-duplex

Port 1 Link Up - speed 10000 Mbps - full-duplex

IPSEC: entering main loop on lcore 0

IPSEC:  -- lcoreid=0 portid=0 rxqueueid=0

IPSEC:  -- lcoreid=0 portid=1 rxqueueid=0

IPSEC_ESP: pktlen 110, esp_len 24, digest_len 12, payload_len 110,
pad_payload_len 112, block_sz 16, pad_len 2

IPSEC_ESP: pktlen 168

qat_req: at [0x7faefcb40000], len=128

00000000: 00 02 04 80 24 00 00 00 50 B4 42 B1 00 00 00 00 | ....$...P.B.....

00000010: 00 00 28 00 00 00 00 00 80 FA A5 75 AF 7F 00 00 | ..(........u....

00000020: 12 FC 25 AA 00 00 00 00 12 FC 25 AA 00 00 00 00 | ..%.......%.....

00000030: A8 00 00 00 A8 00 00 00 2C 00 00 00 70 00 00 00 | ........,...p...

00000040: 85 CD 08 06 74 43 34 53 29 A0 2E 16 B5 0D FC 6A | ....tC4S)......j

00000050: 14 00 00 00 88 00 00 00 00 00 00 00 00 00 00 00 | ................

00000060: AE FC 25 AA 00 00 00 00 00 00 00 0C 02 02 00 21 | ..%............!

00000070: 00 00 10 42 00 00 0C 0C 00 18 15 18 00 00 00 00 | ...B............

IPSEC: Cryptodev 0 queue 0: enqueued 1 packets (out of 1)

IPSEC: Cryptodev 0 queue 0: dequeued 0 packets (out of 32)



Regards,

Chinmaya

On Wed, Jun 15, 2016 at 6:05 PM, Sergio Gonzalez Monroy <
sergio.gonzalez.monroy at intel.com> wrote:

> On 15/06/2016 13:01, Chinmaya Dwibedy wrote:
>
>> Also the output lspci on VM as follows
>>
>> [root at vpn-c openssl-async]# lspci -nn | grep 0435
>>
>> 00:05.0 Co-processor [0b40]: Intel Corporation Coleto Creek PCIe Endpoint
>> [8086:0435]
>>
>>
> This is the PF, not the VF.
>
> I run the  ipsec-secgw sample application (
>> http://dpdk.org/browse/dpdk/tree/examples/ipsec-secgw) on VM with -cdev
>> QAT
>> (e.g., #./examples/ipsec-secgw/build/ipsec-secgw  -l 0,1 -n 4  -- -p 0x3
>> -P
>> --config="(0,0,0), (1,0,1)" --cdev QAT --ep0) . I found that,
>> rte_cryptodev_count_devtype() returns zero (i.e., no QAT crypto device
>> found).
>>
>>
>> Can anyone please suggest what might be the issue and the way to resolve
>> this ?  Also kindly let me know  using QAT device (passed through mode) by
>> DPDK application is possible or not. Thank you in advance for your time
>> and
>> help. please feel free to let me know if additional information is needed.
>>
>>
> I reckon that your issue is that you are trying to use the PF (not
> supported by DPDK)
> instead of the VF.
>
> From
> http://dpdk.readthedocs.io/en/v2.2.0/cryptodevs/qat.html#installation-using-01-org-qat-driver
> :
>
> "You can use|cat/proc/icp_dh895xcc_dev0/version|to confirm the driver is
> correctly installed.
> You can use|lspci-d:443|to confirm the bdf of the 32 VF devices are
> available per|DH895xCC|device."
>
> You should pass-through one of the VFs.
>
> Sergio
>
Previous message: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)
Next message: [dpdk-users] Issue with ipsec-secgw sample application on VM using Intel QAT device (pass-through mode)
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the users mailing list