[dpdk-users] Hard Crash with X710 and Pktgen

[Rosen, Rami]

Hi, James,

First I agree with Keith about that this means something in that configuration is not setup correctly
and DPDK is not finding the ports in the system, and has nothing to do with Pktgen. 
It seems to me that if you will try to run any application (like testpmd) on this setup you will get the same problem.
Can you please check and confirm ? 

Second, I think the root cause for not finding the ports is around this message ("failed to select IOMMU type") in:
...
> EAL:   probe driver: 8086:1572 net_i40e
> EAL:   0000:09:00.0 failed to select IOMMU type
> EAL: Requested device 0000:09:00.0 cannot be used
> EAL: PCI device 0000:09:00.1 on NUMA socket 0
> EAL:   probe driver: 8086:1572 net_i40e
> EAL:   0000:09:00.1 failed to select IOMMU type
> EAL: Requested device 0000:09:00.1 cannot be used
...

What does " find /sys/kernel/iommu_groups/ -type l"
give ? 

Could it be that 0000:09:00.0 and 0000:09:00.1 belongs to an IOMMU group in which there are other devices ? 

Regards,
Rami Rosen


-----Original Message-----
From: users [mailto:users-bounces at dpdk.org] On Behalf Of Wiles, Keith
Sent: Monday, November 13, 2017 23:50
To: James Bensley <jwbensley at gmail.com>
Cc: users at dpdk.org
Subject: Re: [dpdk-users] Hard Crash with X710 and Pktgen



> On Nov 13, 2017, at 11:35 AM, James Bensley <jwbensley at gmail.com> wrote:
> 
> Hi All,
> 
> I have some HPE DL boxes that use "HP" NICs (OEM Intel X710s). I have 
> compiled DPDK & Pktgen but when I try to run Pktgen the box hard 
> crashes (it locks up and I have bounce it using IPMI).
> 
> The main output is below although a full transcript of my attempt is
> here: 
> https://null.53bits.co.uk/uploads/linux/applications/dpdk/CentOS-DL120
> -DPDK-NIC-Bind.txt
> 
> Essentially I can bind the NICs to DPDK using the igb_uio driver, 
> which leads to the hard crash, or I can bind the NICs to DPDK using 
> the vfio-pci driver but then Pktgen reports that there are no NICs 
> available to use and exits.


In Pktgen I just use the normal rte_eth_device_count() (whatever it is called) to find the ports in the system. This means something in that configuration is not setup and DPDK is not finding the ports in the system. Has nothing to do with Pktgen that I know of, but I have not tried vfio-pci driver and can not as I am out of town.

> 
> Does anyone have any ideas whats going on here? What is "failed to 
> select IOMMU type" telling me?


For the hard lockup problem try using testpmd application and see if that the same problem if not then it will be next week before I can look at.

Let me know if testpmd works or not.

> 
> CentOS 7 / Kernel 4.13 / DPDK 17.08 / Pktgen 3.4.2.
> 
> Cheers,
> James.
> 
> 
> $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 2-6 -n 1 -w 09:00.0 
> -w 09:00.1 -v -- -P -m [3-4].0 [5-6].1
> 
> Copyright (c) <2010-2017>, Intel Corporation. All rights reserved.
> Powered by DPDK
> EAL: Detected 16 lcore(s)
> EAL: RTE Version: 'DPDK 17.08.0'
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: PCI device 0000:09:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:1572 net_i40e
> 
> << HARD LOCK-UP >>
> 
> 
> $ sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 2-6 -n 1 -w 09:00.0 
> -w 09:00.1 -v -- -P -m [3-4].0 [5-6].1
> 
> Copyright (c) <2010-2017>, Intel Corporation. All rights reserved.
> Powered by DPDK
> EAL: Detected 16 lcore(s)
> EAL: RTE Version: 'DPDK 17.08.0'
> EAL: No free hugepages reported in hugepages-1048576kB
> EAL: Probing VFIO support...
> EAL: VFIO support initialized
> EAL: PCI device 0000:09:00.0 on NUMA socket 0
> EAL:   probe driver: 8086:1572 net_i40e
> EAL:   0000:09:00.0 failed to select IOMMU type
> EAL: Requested device 0000:09:00.0 cannot be used
> EAL: PCI device 0000:09:00.1 on NUMA socket 0
> EAL:   probe driver: 8086:1572 net_i40e
> EAL:   0000:09:00.1 failed to select IOMMU type
> EAL: Requested device 0000:09:00.1 cannot be used
> 
>   Copyright (c) <2010-2017>, Intel Corporation. All rights reserved.
>   Pktgen created by: Keith Wiles -- >>> Powered by DPDK <<<
> 
>>>> Packet Burst 64, RX Desc 1024, TX Desc 2048, mbufs/port 16384, mbuf 
>>>> cache 2048
> !PANIC!: *** Did not find any ports to use *** PANIC in 
> pktgen_config_ports():
> *** Did not find any ports to use ***6:
> [./app/x86_64-native-linuxapp-gcc/pktgen() [0x44ba76]]
> 5: [/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f16b7238c05]]
> 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x57c) [0x448edc]]
> 3: 
> [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x15b7)
> [0x470d37]]
> 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xb8) 
> [0x4432bc]]
> 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x1a) 
> [0x4b245a]] Aborted

Regards,
Keith