[dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
Zhilong Zheng
zhengzl0715 at gmail.com
Tue Aug 1 11:12:06 CEST 2017
~35Gbps and ~40Gbps when setting to 128 and 256. I use 4 cores to tx
packets.
在 2017年8月1日，17:04，Paul Tsvika <mozloverinweb at gmail.com> 写道：
Hi Zheng,
What throughput you had when you tried 128 or 256B packets?
P.T
2017-08-01 16:53 GMT+08:00 Zhilong Zheng <zhengzl0715 at gmail.com>:
> Hi Paul,
>
> You can refer to the website (https://community.mellanox.co
> m/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
> the way, it looks like that you can not get ~40Gbps throughputs when
> generating 64B packets as @Adrien mentioned (Actually I am also confused
> about this problem before, http://dpdk.org/ml/archives/us
> ers/2017-July/002226.html).
>
> Hope it can help you.
>
> Regards,
>
> Zhilong
>
>
> 在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb at gmail.com> 写道：
>
> Hi zheng,
>
> Thanks for the feedback.
>
> You are right. To get pktgen work with mlx4, it needs to recompile DPDK
> and pktgen as well.
>
> I am able to run pktgen now.
>
> By following the document from DPDK website for this NIC,
> I did and change the following variables
>
> I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
> set l*og_num_mgm_entry_size=-7*
>
> Disable pause frames:
> *ethtool -A ens2d1 rx off tx 0ff*
>
> I did not change the NIC register ( *setpci* ) at this time.
>
>
> and then I run the commands below in order:
>
>
> *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
> *test$: /etc/init.d/openibd restart*
> *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
> --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
> 05:00.1 -- -P -m "[1-5.0]"*
>
> | Ports 0-1 of 2 <Main Page> Copyright (c) <2010-2017>, Intel
> Corporation
> Flags:Port : P--------------:0
> Link State : <--Down--> ----TotalRate----
> Pkts/s Max/Rx : 0/0 0/0
> Max/Tx : 844160/732319 844160/732319
> MBits/s Rx/Tx : 0/*492 * 0/*492*
> Broadcast : 0
> Multicast : 0
> 64 Bytes : 0
> 65-127 : 0
> 128-255 : 0
> 256-511 : 0
> 512-1023 : 0
> 1024-1518 : 0
> Runts/Jumbos : 0/0
> Errors Rx/Tx : 0/0
> Total Rx Pkts : 0
> Tx Pkts : 59289215
> Rx MBs : 0
> Tx MBs : 39842
> ARP/ICMP Pkts : 0/0
> Tx Overrun : 0
> Cycles per Tx : 14880952/3200
> Missed Rx : 0
> mcasts Rx : None
> No Mbuf Rx : 0
> :
> Pattern Type : abcd...
> Tx Count/% Rate : Forever /100%
> PktSize/Tx Burst : 64 / 32
> Src/Dest Port : 1234 / 5678
> Pkt Type:VLAN ID : IPv4 / TCP:0001
> Dst IP Address : 192.168.1.1
> Src IP Address : 192.168.0.1/24
> Src MAC Address : e4:1d:2d:02:44:81
> Powered by DPDK --------------------------
>
> The transmitting rate should be around 40000 since it is 40G NIC.
> However, the rate is much lower than it and it is probably related to
> configurations.
>
> I actually have two mother boards and each one has 40G NICs installed.
> My intention is to do the tx / rx experiment.
> My first step is to check tx rate can reach as close to 40000 as possible.
>
> Anyone has related experience please advise.
>
>
>
> P.T
>
>
> 2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715 at gmail.com>:
>
>>
>> Hi,
>>
>> After enabling the MLX4 flag and recompiling the DPDK source code, Do you
>> recompile the code of pktgen? I think you can recompile it by running make
>> clean and make.
>>
>> Regards,
>> Zhilong
>>
>> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb at gmail.com> 写道：
>> >
>> > Hi Olga,
>> >
>> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas at mellanox.com>:
>> >
>> >> Hi Paul,
>> >>
>> >> Looks like your ConnectX-3 configured to be IB and not ETH
>> >>
>> >>> pci at 0000:04:00.0 ib0 network MT27500 Family [ConnectX-3]
>> >>
>> >> Use connectx_port_config script.
>> >> Set all the ports to Ethernet, and follow the instructions on the
>> screen.
>> >>
>> >
>> > Thanks. I followed the script and configured both ports to Ethernet.
>> >
>> > ibstat info below:
>> >
>> > CA 'mlx4_0'
>> > CA type: MT4099
>> > Number of ports: 2
>> > Firmware version: 2.40.7000
>> > Hardware version: 1
>> > Node GUID: 0xe41d2d0300024480
>> > System image GUID: 0xe41d2d0300024483
>> > Port 1:
>> > State: Down
>> > Physical state: Disabled
>> > Rate: 10
>> > Base lid: 0
>> > LMC: 0
>> > SM lid: 0
>> > Capability mask: 0x04010000
>> > Port GUID: 0xe61d2dfffe024481
>> > Link layer: Ethernet
>> > Port 2:
>> > State: Active
>> > Physical state: LinkUp
>> > Rate: 40
>> > Base lid: 0
>> > LMC: 0
>> > SM lid: 0
>> > Capability mask: 0x04010000
>> > Port GUID: 0xe61d2dfffe024482
>> > Link layer: Ethernet
>> >
>> > I also change the default value from n to CONFIG_RTE_LIBRTE_MLX4_PMD=y.
>> > Recompiled DPDK and it all worked.
>> >
>> > And Below is my ifconfig info:
>> >
>> > ens2 Link encap:Ethernet HWaddr e4:1d:2d:02:44:81
>> > UP BROADCAST MULTICAST MTU:1500 Metric:1
>> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> > collisions:0 txqueuelen:1000
>> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>> >
>> > ens2d1 Link encap:Ethernet HWaddr e4:1d:2d:02:44:82
>> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
>> > RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>> > TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>> > collisions:0 txqueuelen:1000
>> > RX bytes:29494 (29.4 KB) TX bytes:38063 (38.0 KB)
>> >
>> >
>> > Since I assume that I don't have to bind the device manually, I executed
>> > ./dpdk-devbind.py --status with the following info popped out:
>> >
>> > Network devices using kernel driver
>> > ===================================
>> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
>> > drv=ixgbe unused=
>> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
>> > drv=ixgbe unused=
>> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
>> > drv=mlx4_core unused=
>> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
>> unused=
>> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
>> unused=
>> > *Active*
>> >
>> >
>> > With these configurations, I run commands in order but still had the
>> same
>> > result:
>> >
>> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>> > $test: /etc/init.d/openibd restart
>> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>> --proc-type
>> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P
>> -m
>> > "[1-5.0]"
>> >
>> >
>> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
>> cache
>> > 1024
>> > !PANIC!: *** Did not find any ports to use ***
>> > PANIC in pktgen_config_ports():
>> > *** Did not find any ports to use ***6:
>> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
>> > [0x447239]]
>> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
>> > [0x7f0ec0931830]]
>> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
>> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
>> > [0x46e324]]
>> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
>> [0x43ead4]]
>> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
>> [0x4b7ecb]]
>> > Aborted (core dumped)
>> >
>> >
>> >
>> > Is there anything I've done wrong here?
>> >
>> >
>> > Thanks
>> >
>> >
>> > P.T
>> >
>> >
>> >> You can use -w to specify ConnectX-3 PCI address, you don't need to
>> use -b
>> >>
>> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>> >>
>> >> Best Regards,
>> >> Olga
>> >>
>> >>
>> >>
>> >
>> > --
>> > P.T
>>
>
>
>
> --
> P.T
>
>
>
--
P.T
More information about the users
mailing list