[dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors

Paul Tsvika mozloverinweb at gmail.com
Tue Aug 1 11:37:59 CEST 2017
Previous message: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
Next message: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
I found the issue with the output log when executed pktgen.

| Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel Corporation
  Flags:Port      :   P--------------:0
Link State        :      *    <--Down--> *    ----TotalRate----
Pkts/s Max/Rx     :                 0/0                   0/0


The link is down. This looks not right.


Do you have any idea to check this?




P.T

2017-08-01 17:12 GMT+08:00 Zhilong Zheng <zhengzl0715 at gmail.com>:

> ~35Gbps and ~40Gbps when setting to 128 and 256. I use 4 cores to tx
> packets.
>
> 在 2017年8月1日，17:04，Paul Tsvika <mozloverinweb at gmail.com> 写道：
>
> Hi Zheng,
>
> What throughput you had when you tried 128 or 256B packets?
>
>
> P.T
>
> 2017-08-01 16:53 GMT+08:00 Zhilong Zheng <zhengzl0715 at gmail.com>:
>
>> Hi Paul,
>>
>> You can refer to the website (https://community.mellanox.co
>> m/docs/DOC-2489#jive_content_id_Getting_started) to tune performance. By
>> the way, it looks like that you can not get ~40Gbps throughputs when
>> generating 64B packets as @Adrien mentioned (Actually I am also confused
>> about this problem before, http://dpdk.org/ml/archives/us
>> ers/2017-July/002226.html).
>>
>> Hope it can help you.
>>
>> Regards,
>>
>> Zhilong
>>
>>
>> 在 2017年8月1日，16:15，Paul Tsvika <mozloverinweb at gmail.com> 写道：
>>
>> Hi zheng,
>>
>> Thanks for the feedback.
>>
>> You are right. To get pktgen work with mlx4, it needs to recompile DPDK
>> and pktgen as well.
>>
>> I am able to run pktgen now.
>>
>> By following the document from DPDK website for this NIC,
>> I did and change the following variables
>>
>> I set *MLX4_INLINE_RECV_SIZE=64* as the environment variable in bashrc.
>> set l*og_num_mgm_entry_size=-7*
>>
>> Disable pause frames:
>> *ethtool -A ens2d1 rx off tx 0ff*
>>
>> I did not change the NIC register ( *setpci* ) at this time.
>>
>>
>> and then I run the commands below in order:
>>
>>
>> *test$: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib*
>> *test$: /etc/init.d/openibd restart*
>> *test:$: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>> --proc-type auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b
>> 05:00.1 -- -P -m "[1-5.0]"*
>>
>> | Ports 0-1 of 2   <Main Page>  Copyright (c) <2010-2017>, Intel
>> Corporation
>>   Flags:Port      :   P--------------:0
>> Link State        :          <--Down-->     ----TotalRate----
>> Pkts/s Max/Rx     :                 0/0                   0/0
>>        Max/Tx     :       844160/732319         844160/732319
>> MBits/s Rx/Tx     :               0/*492 *                0/*492*
>> Broadcast         :                   0
>> Multicast         :                   0
>>   64 Bytes        :                   0
>>   65-127          :                   0
>>   128-255         :                   0
>>   256-511         :                   0
>>   512-1023        :                   0
>>   1024-1518       :                   0
>> Runts/Jumbos      :                 0/0
>> Errors Rx/Tx      :                 0/0
>> Total Rx Pkts     :                   0
>>       Tx Pkts     :            59289215
>>       Rx MBs      :                   0
>>       Tx MBs      :               39842
>> ARP/ICMP Pkts     :                 0/0
>> Tx Overrun        :                   0
>> Cycles per Tx     :       14880952/3200
>> Missed Rx         :                   0
>> mcasts Rx         :                None
>> No Mbuf Rx        :                   0
>>                   :
>> Pattern Type      :             abcd...
>> Tx Count/% Rate   :       Forever /100%
>> PktSize/Tx Burst  :           64 /   32
>> Src/Dest Port     :         1234 / 5678
>> Pkt Type:VLAN ID  :     IPv4 / TCP:0001
>> Dst  IP Address   :         192.168.1.1
>> Src  IP Address   :      192.168.0.1/24
>> Src MAC Address   :   e4:1d:2d:02:44:81
>> Powered by DPDK --------------------------
>>
>> The transmitting rate should be around 40000 since it is 40G NIC.
>> However, the rate is much lower than it and it is probably related to
>> configurations.
>>
>> I actually have two mother boards and each one has 40G NICs installed.
>> My intention is to do the tx / rx experiment.
>> My first step is to check tx rate can reach as close to 40000 as possible.
>>
>> Anyone has related experience please advise.
>>
>>
>>
>> P.T
>>
>>
>> 2017-08-01 12:15 GMT+08:00 zhilong zheng <zhengzl0715 at gmail.com>:
>>
>>>
>>> Hi,
>>>
>>> After enabling the MLX4 flag and recompiling the DPDK source code, Do
>>> you recompile the code of pktgen? I think you can recompile it by running
>>> make clean and make.
>>>
>>> Regards,
>>> Zhilong
>>>
>>> > 在 2017年8月1日，11:38，Paul Tsvika <mozloverinweb at gmail.com> 写道：
>>> >
>>> > Hi Olga,
>>> >
>>> > 2017-08-01 0:11 GMT+08:00 Olga Shern <olgas at mellanox.com>:
>>> >
>>> >> Hi Paul,
>>> >>
>>> >> Looks like your ConnectX-3 configured to be IB and not ETH
>>> >>
>>> >>> pci at 0000:04:00.0  ib0     network        MT27500 Family [ConnectX-3]
>>> >>
>>> >> Use connectx_port_config script.
>>> >> Set all the ports to Ethernet, and follow the instructions on the
>>> screen.
>>> >>
>>> >
>>> > Thanks. I followed the script and configured  both ports to Ethernet.
>>> >
>>> > ibstat info below:
>>> >
>>> > CA 'mlx4_0'
>>> >    CA type: MT4099
>>> >    Number of ports: 2
>>> >    Firmware version: 2.40.7000
>>> >    Hardware version: 1
>>> >    Node GUID: 0xe41d2d0300024480
>>> >    System image GUID: 0xe41d2d0300024483
>>> >    Port 1:
>>> >        State: Down
>>> >        Physical state: Disabled
>>> >        Rate: 10
>>> >        Base lid: 0
>>> >        LMC: 0
>>> >        SM lid: 0
>>> >        Capability mask: 0x04010000
>>> >        Port GUID: 0xe61d2dfffe024481
>>> >        Link layer: Ethernet
>>> >    Port 2:
>>> >        State: Active
>>> >        Physical state: LinkUp
>>> >        Rate: 40
>>> >        Base lid: 0
>>> >        LMC: 0
>>> >        SM lid: 0
>>> >        Capability mask: 0x04010000
>>> >        Port GUID: 0xe61d2dfffe024482
>>> >        Link layer: Ethernet
>>> >
>>> > I also change the default value from n to
>>> CONFIG_RTE_LIBRTE_MLX4_PMD=y.
>>> > Recompiled DPDK and it all worked.
>>> >
>>> > And Below is my ifconfig info:
>>> >
>>> > ens2      Link encap:Ethernet  HWaddr e4:1d:2d:02:44:81
>>> >          UP BROADCAST MULTICAST  MTU:1500  Metric:1
>>> >          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>>> >          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>>> >          collisions:0 txqueuelen:1000
>>> >          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
>>> >
>>> > ens2d1    Link encap:Ethernet  HWaddr e4:1d:2d:02:44:82
>>> >          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>> >          RX packets:126 errors:0 dropped:0 overruns:0 frame:0
>>> >          TX packets:220 errors:0 dropped:0 overruns:0 carrier:0
>>> >          collisions:0 txqueuelen:1000
>>> >          RX bytes:29494 (29.4 KB)  TX bytes:38063 (38.0 KB)
>>> >
>>> >
>>> > Since I assume that I don't have to bind the device manually, I
>>> executed
>>> > ./dpdk-devbind.py --status with the following info popped out:
>>> >
>>> > Network devices using kernel driver
>>> > ===================================
>>> > 0000:03:00.0 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno3
>>> > drv=ixgbe unused=
>>> > 0000:03:00.1 'Ethernet Connection X552/X557-AT 10GBASE-T 15ad' if=eno4
>>> > drv=ixgbe unused=
>>> > 0000:04:00.0 'MT27500 Family [ConnectX-3] 1003' if=ens2,ens2d1
>>> > drv=mlx4_core unused=
>>> > 0000:05:00.0 'I350 Gigabit Network Connection 1521' if=eno1 drv=igb
>>> unused=
>>> > 0000:05:00.1 'I350 Gigabit Network Connection 1521' if=eno2 drv=igb
>>> unused=
>>> > *Active*
>>> >
>>> >
>>> > With these configurations, I run commands in order but still had the
>>> same
>>> > result:
>>> >
>>> > $test: modprobe -a ib_uverbs mlx4_en mlx4_core mlx4_ib
>>> > $test: /etc/init.d/openibd restart
>>> > $test: sudo ./app/x86_64-native-linuxapp-gcc/pktgen -l 0-6 -n 1
>>> --proc-type
>>> > auto --file-prefix pg -b 03:00.0 -b 03:00.1 -b 05:00.0 -b 05:00.1 --
>>> -P -m
>>> > "[1-5.0]"
>>> >
>>> >
>>> >>>> Packet Burst 32, RX Desc 512, TX Desc 1024, mbufs/port 8192, mbuf
>>> cache
>>> > 1024
>>> > !PANIC!: *** Did not find any ports to use ***
>>> > PANIC in pktgen_config_ports():
>>> > *** Did not find any ports to use ***6:
>>> > [./app/x86_64-native-linuxapp-gcc/pktgen(_start+0x29)
>>> > [0x447239]]
>>> > 5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)
>>> > [0x7f0ec0931830]]
>>> > 4: [./app/x86_64-native-linuxapp-gcc/pktgen(main+0x580) [0x444710]]
>>> > 3: [./app/x86_64-native-linuxapp-gcc/pktgen(pktgen_config_ports
>>> +0x1614)
>>> > [0x46e324]]
>>> > 2: [./app/x86_64-native-linuxapp-gcc/pktgen(__rte_panic+0xc3)
>>> [0x43ead4]]
>>> > 1: [./app/x86_64-native-linuxapp-gcc/pktgen(rte_dump_stack+0x2b)
>>> [0x4b7ecb]]
>>> > Aborted (core dumped)
>>> >
>>> >
>>> >
>>> > Is there anything I've done wrong here?
>>> >
>>> >
>>> > Thanks
>>> >
>>> >
>>> > P.T
>>> >
>>> >
>>> >> You can use  -w to specify ConnectX-3 PCI address, you don't need to
>>> use -b
>>> >>
>>> >> Make sure you compile DPDK with mlx4 PMD, it is not enabled by default
>>> >>
>>> >> Best Regards,
>>> >> Olga
>>> >>
>>> >>
>>> >>
>>> >
>>> > --
>>> > P.T
>>>
>>
>>
>>
>> --
>> P.T
>>
>>
>>
>
>
> --
> P.T
>
>
>


-- 
P.T
Previous message: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
Next message: [dpdk-users] [dpdk][pktgen] - execute pktgen with mlx4 (40G) with some errors
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the users mailing list