[dpdk-dev] DPDK-pktgen not giving expected throughput
Wiles, Keith
keith.wiles at intel.com
Thu Jan 25 16:39:11 CET 2018
> On Jan 25, 2018, at 1:56 AM, Shailja Pandey <csz168117 at iitd.ac.in> wrote:
>
> Hi,
>
> I tried this command line and blacklisted 04:00.0 and 04:00.1-
>
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -b 04:00.0 -b 04:00.1 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”
> But I am still not able to see the ports using comma separated cores. I have also tried the following configuration but not getting 100 Gbps for packet size of 64 bytes and 1500 bytes:
>
> make; ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffffffff -n 4 -b 04:00.0 -b 04:00.1 -- -T -P -m "[2-6:7-11].0,[12-16:17-21].1”
I guess I am not completely following the problem are you stating the comma list of cores does not work or just not able to get the performance you want?
Many factors effect the performance like the PCI slot width, location of PCI slot relative to the CPU socket, cores being used are not on the same CPU socket as the PCI slot/card, not setting up the range command to allow RSS to distribute the packets across the RX ports.
We need to isolate and verify these possible problems could be the issue.
>
> Any help would be highly appreciated.
> > On Jan 13, 2018, at 12:51 PM, Shailja Pandey <csz168117 at iitd.ac.in
> > wrote:
>
> >
>
>
> >
> Hi,
>
> >
>
>
> >
> Output of command ‘lspci | grep Ether’ is-
>
> >
>
>
> >
> 04:00.0 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
>
> >
> 04:00.1 Ethernet controller: Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 (rev 01)
>
> >
> 05:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
>
> >
> 05:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02)
>
> >
>
>
> >
> I tried running pktgen with this command
>
> >
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”
>
>
> Need to blacklist 04:00.0 and 04:00.1 using -b 04:00.0 -b 04:00.1 just before the '--'
>
> >
>
>
> >
> but I am unable to see the ports while using comma separated cores like 2,4,6. However when I tried with multiple CPUs, the result is still the same.
>
> >
>
>
> >
> After allocating multiple cores, I also tried the range commands to distribute the packets on different cores(RSS).
>
> >
>
>
> >
>
>
> >
> >
>
> >
> On Jan 11, 2018, at 12:45 AM, Shailja Pandey <csz168117 at iitd.ac.in
>
> >
> > wrote:
>
> >
>
>
> >
> >
>
> >
>
>
> >
>
>
> >
> >
>
> >
> I have also tried with multiple cpus using the command line-
>
> >
>
>
> >
> >
>
> >
>
>
> >
>
>
> >
> >
>
> >
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m "[0:4].0,[5:8].1,[9-12].2,[13-16].3”
>
> >
>
>
> >
>
>
> >
> What is port 0 and 1, I guess that port 2 and 3 are the 2x40 card. I guess I need the ‘lspci | grep Ether’ output too.
>
> >
>
>
> >
> Lets assume port 2 and 3 are the 2 ports of 40G and the NIC is on PCI bus0 attached to NUMA node 0.
>
> >
>
>
> >
> May need to blacklist the first two port 0 and 1 to remove them from being used.
>
> >
>
>
> >
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xffff -n 4 -- -T -P -m “[2,4,6:8,10,12].0, [14,16,18:20,22,24].1”
>
> >
>
>
> >
> Now you have 6 cores per port 3 on TX and 3 on RX on each port. You maybe be able to get away with 4 per port with the speed of the machine you have.
>
> >
>
>
> >
> Next we have to use the range command to make sure the packets are transmitted with a varying set of 5 tuples.
>
> >
>
>
> >
> Do a help command an look at the range commands, there is an example in the test directory.
>
> >
>
>
> >
> To see the range information use ‘page range’ and then you can modify each port.
>
> >
>
>
> >
> After you are happy with the configuration use the ‘save <filename>’ command to save the configuration. Then you can use ‘load <filename>’ or add it to the command line with the -f <filename> option.
>
> >
>
>
> >
>
>
> >
>
>
> >
>
>
> >
> >
>
> >
>
>
> >
>
>
> >
> >
>
> >
> On Thursday 11 January 2018 10:20 AM, Shailja Pandey wrote:
>
> >
>
>
> >
> >>
>
> >
> The command line for Pktgen is-
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xfff -n 4 -- -T -P -m "[2].0,[4].1,[8].2,[10].3"
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Configuration of the machine is-
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Architecture: x86_64
>
> >
>
>
> >
> >>
>
> >
> CPU op-mode(s): 32-bit, 64-bit
>
> >
>
>
> >
> >>
>
> >
> Byte Order: Little Endian
>
> >
>
>
> >
> >>
>
> >
> CPU(s): 32
>
> >
>
>
> >
> >>
>
> >
> On-line CPU(s) list: 0-31
>
> >
>
>
> >
> >>
>
> >
> Thread(s) per core: 2
>
> >
>
>
> >
> >>
>
> >
> Core(s) per socket: 8
>
> >
>
>
> >
> >>
>
> >
> Socket(s): 2
>
> >
>
>
> >
> >>
>
> >
> NUMA node(s): 2
>
> >
>
>
> >
> >>
>
> >
> Vendor ID: GenuineIntel
>
> >
>
>
> >
> >>
>
> >
> CPU family: 6
>
> >
>
>
> >
> >>
>
> >
> Model: 63
>
> >
>
>
> >
> >>
>
> >
> Model name: Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
>
> >
>
>
> >
> >>
>
> >
> Stepping: 2
>
> >
>
>
> >
> >>
>
> >
> CPU MHz: 1204.226
>
> >
>
>
> >
> >>
>
> >
> CPU max MHz: 3400.0000
>
> >
>
>
> >
> >>
>
> >
> CPU min MHz: 1200.0000
>
> >
>
>
> >
> >>
>
> >
> BogoMIPS: 5195.08
>
> >
>
>
> >
> >>
>
> >
> Virtualization: VT-x
>
> >
>
>
> >
> >>
>
> >
> L1d cache: 32K
>
> >
>
>
> >
> >>
>
> >
> L1i cache: 32K
>
> >
>
>
> >
> >>
>
> >
> L2 cache: 256K
>
> >
>
>
> >
> >>
>
> >
> L3 cache: 20480K
>
> >
>
>
> >
> >>
>
> >
> NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30
>
> >
>
>
> >
> >>
>
> >
> NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Linux XeonE5 4.4.0-93-generic
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Thanks!
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
> On Jan 10, 2018, at 7:36 AM, Shailja Pandey <csz168117 at iitd.ac.in
>
> >
>
>
> >
> >>
>
> >
> > wrote:
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
> Hi,
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
> We are performing experiments on Dell Poweredge R430 server, which is based on Haswell architecture based xeon-2640 v3 processor. We have attached XL 710 NIC(2x40 GbE) to the machine and expect 59 Mpps packet generation per port. But pktgen is able to generate only 21 Mpps per port and we are not sure about the problem.
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
> We are using DPDK version 16.07 and DPDK-pktgen version 3.1.0. Is there any way to generate packets from pktgen at 118 Mpps ?
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> What is the command line for Pktgen?
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Using a single CPU you should be able to generate about 21Mpps, which tells me the configuration is not correct.
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Pktgen needs N number of TX and RX cores to generate more traffic, plus the packets need to be spread across the cores using RSS. This means pktgen needs to be setup to generate packets with the 5 tuple difference to get all of the cores to receive the packets.
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> I need to know the command line and the configuration of the machine to help.
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
> --
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
> Thanks,
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
> Shailja
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> >
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Regards,
>
> >
>
>
> >
> >>
>
> >
> Keith
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> --
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >>
>
> >
> Thanks,
>
> >
>
>
> >
> >>
>
> >
> Shailja
>
> >
>
>
> >
> >>
>
> >
>
>
> >
>
>
> >
> >
>
> >
>
>
> >
>
>
> >
> >
>
> >
> --
>
> >
>
>
> >
> >
>
> >
>
>
> >
>
>
> >
> >
>
> >
> Thanks,
>
> >
>
>
> >
> >
>
> >
> Shailja
>
> >
>
>
> >
> >
>
> >
>
>
> >
>
>
> >
>
>
> >
> Regards,
>
> >
> Keith
>
> >
>
>
> >
> --
>
> >
>
>
> >
> Thanks,
>
> >
> Shailja
>
> >
>
>
>
> Regards,
> Keith
>
> --
>
> Thanks,
> Shailja
>
> --
>
> Thanks,
> Shailja
>
Regards,
Keith
More information about the dev
mailing list