[dpdk-users] L2fwd very slow throughput

[Simon Jouet]

Hi everyone,

I went through previous messages on the mailing list to look for people with a similar issue, even though I found some users with slower than expected throughput it is still significantly higher than what I’m getting (http://dpdk.org/ml/archives/dev/2014-July/004114.html). Therefore I would like to know if some people have experienced similar problems or would be able to point out what might be wrong with the setup.

The setup is fairly standard, I have two hosts with Intel X710 quad port 10G NICs. One host is running l2fwd and the other one is running MoonGen with the rfc2544 throughput test. The two hosts are connected with two SFP+ cables on port 0 and 1 of the NICs.

Both machines are similar, with an Intel 6700K Skylake processor (4 cores at 4GHz with hyperthreading) and 32GB of DDR4 RAM (clocked at 3GHz) running Linux with a kernel 4.4.3 and 8, 1G hugepages and finally on both hosts DPDK 2.2.0 is used and the uio_pci_generic driver is loaded. I think that’s it for the setup.

If I run the throughput test using the Linux bridge with 64 bytes packet size, I achieve a slow but expected throughput of 1505.14Mbit/s

       maximal rate for packetsize 64: 2.24 Mpps, 1146.77 MBit/s, 1505.14 MBit/s wire rate

Running the same test with l2fwd instead of the Linux bridge ends up with a very slow throughput

       maximal rate for packetsize 64: 1.07 Mpps, 546.08 MBit/s, 716.74 MBit/s wire rate

L2Fwd is started using the command below, I tried to give more or less cores, change the number of memory channel and also the number of RX queues but the result is always pretty much exactly the same.
       sudo ./build/l2fwd -c 0xff -n 3 -- -p 3

I tried to use testpmd in iofwd and macfwd mode with the default configuration and with exactly the same test I reach the 10Gbit/s without problem however with the l2fwd, l3fwd the performances are slow as described above. I tried to change the rx buffer size in the l2fwd example from 32 to 64 but the result is the same.

Finally I tried the netmap_compat bridge and the result is half the speed of l2fwd

       maximal rate for packetsize 64: 0.75 Mpps, 382.26 MBit/s, 501.71 MBit/s wire rate

Am I missing something obvious?

Best regards,
Simon