[dpdk-dev] Performances are not scale with multiple ports

[Naoto MATSUMOTO]

Hi shiane and all

Emre and Stephen, thanks your suggest for us.

We are not testing multi-port traffic gen using NETMAP though DPDK l3fw,
but we understood some packet recive-side without NETMAP Linux.

Please try it using stephen's pktgen if you have more traffic gen/recive
server resources or Using IXIA.

best regards,
--
Naoto



On Tue, 28 May 2013 12:05:50 +0900
Shinae Woo <shinae2012 at gmail.com> wrote:

> Thanks for sharing Naoto.
> 
> So in your experiments, the forwarding performance still does not reach the
> line rate.
> 
> Your perf record shows that the cpu spend most of the time in polling for
> receiving packets,
> and no other heavy operation.
> Even though the application polling packets in its best,
> the forwarder miss some packets from elsewhere from the application-side.
> 
> The dpdk document shows that 160Mpps forwarding performance in 2 sockets,
> but I can only reach the 13 Mpps in 2 ports.
> Even doubling the number of ports to 4 ports, the performance is still less
> than 17Mpps.
> 
> I want to know where is the bottleneck lies in my environments, or
> how I can reprocuce the same performance as the dpdk published.
> 
> Thank you,
> Shinae
> 
> 
> 
> On Tue, May 28, 2013 at 11:30 AM, Naoto MATSUMOTO
> <n-matsumoto at sakura.ad.jp>wrote:
> 
> >
> > FYI: Disruptive IP Networking with Intel DPDK on Linux
> > http://slidesha.re/SeVFZo
> >
> >
> > On Tue, 28 May 2013 11:26:30 +0900
> > Shinae Woo <shinae2012 at gmail.com> wrote:
> >
> > > Hello, all.
> > >
> > > I play the dpdk-1.2.3r1 with examples.
> > >
> > > But I can not achieve the line-rate packet receive performance,
> > > and the performance is not scale with multiple ports.
> > >
> > > For example, in example l2fwd, I have tested two cases with 2 ports, and
> > 4
> > > ports,
> > > using belowed command line each
> > >
> > > ./build/l2fwd -cf -n3 -- -p3
> > > ./build/l2fwd -cf -n3 -- -pf
> > >
> > > But both cases, the aggregated performance are not scale.
> > >
> > > == experiments environments ==
> > > - Two Intel 82599 NICs (total 4 ports)
> > > - Intel Xeon X5690  @ 3.47GHz * 2 (total 12 cores)
> > > - 1024 * 2MB hugepages
> > > - Linux 2.6.38-15-server
> > > - Each ports receiving 10Gbps of traffic of 64 bytes packets, 14.88Mpps.
> > >
> > > *1. Packet forwarding performance*
> > >
> > > In 2 ports case,  receive performance is 13Mpps,
> > > In 4 ports case,  not 26Mbps, only 16.8Mpps.
> > >
> > > Port statistics ====================================
> > > Statistics for port 0 ------------------------------
> > > Packets sent:                  4292256
> > > Packets received:              6517396
> > > Packets dropped:               2224776
> > > Statistics for port 1 ------------------------------
> > > Packets sent:                  4291840
> > > Packets received:              6517044
> > > Packets dropped:               2225556
> > > Aggregate statistics ===============================
> > > Total packets sent:            8584128
> > > Total packets received:       13034472
> > > Total packets dropped:         4450332
> > > ====================================================
> > >
> > > Port statistics ====================================
> > > Statistics for port 0 ------------------------------
> > > Packets sent:                  1784064
> > > Packets received:              2632700
> > > Packets dropped:                848128
> > > Statistics for port 1 ------------------------------
> > > Packets sent:                  1784104
> > > Packets received:              2632196
> > > Packets dropped:                848596
> > > Statistics for port 2 ------------------------------
> > > Packets sent:                  3587616
> > > Packets received:              5816344
> > > Packets dropped:               2200176
> > > Statistics for port 3 ------------------------------
> > > Packets sent:                  3587712
> > > Packets received:              5787848
> > > Packets dropped:               2228684
> > > Aggregate statistics ===============================
> > > Total packets sent:           10743560
> > > Total packets received:       16869152
> > > Total packets dropped:         6125608
> > > ====================================================
> > >
> > > *2. Packet receiving performance*
> > > I fix the codes for only receiving packets (not forwarding),
> > > the performance is still not scalable as each 13.3Mpps, 18Mpps.
> > >
> > > Port statistics ====================================
> > > Statistics for port 0 ------------------------------
> > > Packets sent:                        0
> > > Packets received:              6678860
> > > Packets dropped:                     0
> > > Statistics for port 1 ------------------------------
> > > Packets sent:                        0
> > > Packets received:              6646120
> > > Packets dropped:                     0
> > > Aggregate statistics ===============================
> > > Total packets sent:                  0
> > > Total packets received:       13325012
> > > Total packets dropped:               0
> > > ====================================================
> > >
> > > Port statistics ====================================
> > > Statistics for port 0 ------------------------------
> > > Packets sent:                        0
> > > Packets received:              3129624
> > > Packets dropped:                     0
> > > Statistics for port 1 ------------------------------
> > > Packets sent:                        0
> > > Packets received:              3131292
> > > Packets dropped:                     0
> > > Statistics for port 2 ------------------------------
> > > Packets sent:                        0
> > > Packets received:              6260908
> > > Packets dropped:                     0
> > > Statistics for port 3 ------------------------------
> > > Packets sent:                        0
> > > Packets received:              6238764
> > > Packets dropped:                     0
> > > Aggregate statistics ===============================
> > > Total packets sent:                  0
> > > Total packets received:       18760640
> > > Total packets dropped:               0
> > > ====================================================
> > >
> > > The question is that
> > > 1. How I can achieve each port receiving full 14.88Mpps ?
> > >     What might be the bottleneck in current environment?
> > > 2. Why the performance using multiple ports is not scale?
> > >     I guess doubling ports shows the doubling the receiving performance,
> > >     but it shows not. I am curious about what is limiting the packet
> > > receivng performance.
> > >
> > > Thanks,
> > > Shinae
> >
> > --
> > SAKURA Internet Inc. / Senior Researcher
> > Naoto MATSUMOTO <n-matsumoto at sakura.ad.jp>
> > SAKURA Internet Research Center <http://research.sakura.ad.jp/>
> >
> >

-- 
SAKURA Internet Inc. / Senior Researcher
Naoto MATSUMOTO <n-matsumoto at sakura.ad.jp>
SAKURA Internet Research Center <http://research.sakura.ad.jp/>