[dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
Zhilong Zheng
zhengzl0715 at gmail.com
Tue Aug 1 10:53:11 CEST 2017
Hi Paul,
You can refer to the website (https://community.mellanox.
com/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
the way, it looks like that you can not get ~40Gbps throughputs when
generating 64B packets as @Adrien mentioned (Actually I am also confused
about this problem before, http://dpdk.org/ml/archives/
users/2017-July/002226.html).
Hope it can help you.
Regards,
Zhilong
在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb at gmail.com> 写道：
Hi zheng,
Thanks for the feedback.
You are right. To get pktgen work with mlx4, it needs to recompile DPDK and
pktgen as well.
I am able to run pktgen now.
By following the document from DPDK website for this NIC,
I did and change the following variables
I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
set l*og_num_mgm_entry_size=-7*
Disable pause frames:
*ethtool -A ens2d1 rx off tx 0ff*
I did not change the NIC register ( *setpci* ) at this time.
and then I run the commands below in order:
*test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
*test$: /etc/init.d/openibd restart*
*test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
--proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
05:00.1 -- -P -m "[1-5.0]"*
| Ports 0-1 of 2 <Main Page> Copyright (c) <2010-2017>, Intel Corporation
Flags:Port : P--------------:0
Link State : <--Down--> ----TotalRate----
Pkts/s Max/Rx : 0/0 0/0
Max/Tx : 844160/732319 844160/732319
MBits/s Rx/Tx : 0/*492 * 0/*492*
Broadcast : 0
Multicast : 0
64 Bytes : 0
65-127 : 0
128-255 : 0
256-511 : 0
512-1023 : 0
1024-1518 : 0
Runts/Jumbos : 0/0
Errors Rx/Tx : 0/0
Total Rx Pkts : 0
Tx Pkts : 59289215
Rx MBs : 0
Tx MBs : 39842
ARP/ICMP Pkts : 0/0
Tx Overrun : 0
Cycles per Tx : 14880952/3200
Missed Rx : 0
mcasts Rx : None
No Mbuf Rx : 0
:
Pattern Type : abcd...
Tx Count/% Rate : Forever /100%
PktSize/Tx Burst : 64 / 32
Src/Dest Port : 1234 / 5678
Pkt Type:VLAN ID : IPv4 / TCP:0001
Dst IP Address : 192.168.1.1
Src IP Address : 192.168.0.1/24
Src MAC Address : e4:1d:2d:02:44:81
Powered by DPDK --------------------------
The transmitting rate should be around 40000 since it is 40G NIC.
However, the rate is much lower than it and it is probably related to
configurations.
I actually have two mother boards and each one has 40G NICs installed.
My intention is to do the tx / rx experiment.
My first step is to check tx rate can reach as close to 40000 as possible.
Anyone has related experience please advise.
P.T
2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715 at gmail.com>:
>
> Hi,
>
> After enabling the MLX4 flag and recompiling the DPDK source code, Do you
> recompile the code of pktgen? I think you can recompile it by running make
> clean and make.
>
> Regards,
> Zhilong
>
> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb at gmail.com> 写道：
> >
> > Hi Olga,
> >
> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas at mellanox.com>:
> >
> >> Hi Paul,
> >>
> >> Looks like your ConnectX-3 configured to be IB and not ETH
> >>
> >>> pci at 0000:04:00.0 ib0 network MT27500 Family [ConnectX-3]
> >>
> >> Use connectx_port_config script.
> >> Set all the ports to Ethernet, and follow the instructions on the
> screen.
> >>
> >
> > Thanks. I followed the script and configured both ports to Ethernet.
> >
> > ibstat info below:
> >
> > CA 'mlx4_0'
> > CA type: MT4099
> > Number of ports: 2
> > Firmware version: 2.40.7000
> > Hardware version: 1
> > Node GUID: 0xe41d2d0300024480
> > System image GUID: 0xe41d2d0300024483
> > Port 1:
> > State: Down
> > Physical state: Disabled
> > Rate: 10
> > Base lid: 0
> > LMC: 0
> > SM lid: 0
> > Capability mask: 0x04010000
> > Port GUID: 0xe61d2dfffe024481
> > Link layer: Ethernet
> > Port 2:
> > State: Active
> > Physical state: LinkUp
> > Rate: 40
> > Base lid: 0
> > LMC: 0
> > SM lid: 0
> > Capability mask: 0x04010000
> > Port GUID: 0xe61d2dfffe024482
> > Link layer: Ethernet
> >
> > I also change the default value from n to CONFIG_RTE_LIBRTE_MLX4_PMD=y.
> > Recompiled DPDK and it all worked.
> >
> > And Below is my ifconfig info:
> >
> > ens2 Link encap:Ethernet HWaddr e4:1d:2d:02:44:81
> > UP BROADCAST MULTICAST MTU:1500 Metric:1
> > RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> >
> > ens2d1 Link encap:Ethernet HWaddr e4:1d:2d:02:44:82
> > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> > RX packets:126 errors:0 dropped:0 overruns:0 frame:0
> > TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
> > collisions:0 txqueuelen:1000
> > RX bytes:29494 (29.4 KB) TX bytes:38063 (38.0 KB)
> >
> >
> > Since I assume that I don't have to bind the device manually, I executed
> > ./dpdk-devbind.py --status with the following info popped out:
> >
> > Network devices using kernel driver
> > ===================================
> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
> > drv=ixgbe unused=
> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
> > drv=ixgbe unused=
> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
> > drv=mlx4_core unused=
> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
> unused=
> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
> unused=
> > *Active*
> >
> >
> > With these configurations, I run commands in order but still had the same
> > result:
> >
> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
> > $test: /etc/init.d/openibd restart
> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
> --proc-type
> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 -- -P
> -m
> > "[1-5.0]"
> >
> >
> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
> cache
> > 1024
> > !PANIC!: *** Did not find any ports to use ***
> > PANIC in pktgen_config_ports():
> > *** Did not find any ports to use ***6:
> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
> > [0x447239]]
> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
> > [0x7f0ec0931830]]
> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports+0x1614)
> > [0x46e324]]
> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
> [0x43ead4]]
> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
> [0x4b7ecb]]
> > Aborted (core dumped)
> >
> >
> >
> > Is there anything I've done wrong here?
> >
> >
> > Thanks
> >
> >
> > P.T
> >
> >
> >> You can use -w to specify ConnectX-3 PCI address, you don't need to
> use -b
> >>
> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
> >>
> >> Best Regards,
> >> Olga
> >>
> >>
> >>
> >
> > --
> > P.T
>
--
P.T
More information about the users
mailing list