<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hello Shinae,<br>
<br>
Did you try to use the testpmd tool with multiple queues per port?
It gives you more flexibility compared to l2fwd app.<br>
<br>
You need to trigger the RSS feature of the NIC by sending different
streams (just by changing the destination port for instance or any
information in the 5-tuple). This will load balance your packets
among several cores so that you can probe multiple queues with
different cores. Otherwise, you will use only one core (or thread if
HT is enabled) per port for the RX side.<br>
<br>
Best Regards,<br>
Emre<br>
<pre class="moz-signature" cols="72">--
Emre ERALTAN
6WIND Field Application Engineer
</pre>
<br>
<div class="moz-cite-prefix">On 27/05/2013 20:05, Shinae Woo wrote:<br>
</div>
<blockquote
cite="mid:CA+f=Zzvyfgu8GcdP4R2W6qi0gfEpDhdEctHnhxmqYuTn8U+n3A@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>
<div>
<div>Thanks for sharing Naoto.<br>
<br>
</div>
So in your experiments, the forwarding performance still
does not reach the line rate.<br>
</div>
<br>
Your perf record shows that the cpu spend most of the time in
polling for receiving packets, <br>
</div>
<div>and no other heavy operation.<br>
</div>
<div>Even though the application polling packets in its best,<br>
the forwarder miss some packets from elsewhere from the
application-side.<br>
<br>
</div>
<div>The dpdk document shows that 160Mpps forwarding performance
in 2 sockets,<br>
</div>
<div>but I can only reach the 13 Mpps in 2 ports. <br>
Even doubling the number of ports to 4 ports, the performance
is still less than 17Mpps.<br>
<br>
</div>
<div>I want to know where is the bottleneck lies in my
environments, or<br>
</div>
<div>how I can reprocuce the same performance as the dpdk
published.<br>
<br>
</div>
<div>Thank you,<br>
</div>
<div>Shinae<br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, May 28, 2013 at 11:30 AM, Naoto
MATSUMOTO <span dir="ltr"><<a moz-do-not-send="true"
href="mailto:n-matsumoto@sakura.ad.jp" target="_blank">n-matsumoto@sakura.ad.jp</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
FYI: Disruptive IP Networking with Intel DPDK on Linux<br>
<a moz-do-not-send="true" href="http://slidesha.re/SeVFZo"
target="_blank">http://slidesha.re/SeVFZo</a><br>
<div class="im"><br>
<br>
On Tue, 28 May 2013 11:26:30 +0900<br>
Shinae Woo <<a moz-do-not-send="true"
href="mailto:shinae2012@gmail.com">shinae2012@gmail.com</a>>
wrote:<br>
<br>
> Hello, all.<br>
><br>
> I play the dpdk-1.2.3r1 with examples.<br>
><br>
> But I can not achieve the line-rate packet receive
performance,<br>
> and the performance is not scale with multiple ports.<br>
><br>
> For example, in example l2fwd, I have tested two
cases with 2 ports, and 4<br>
> ports,<br>
> using belowed command line each<br>
><br>
> ./build/l2fwd -cf -n3 -- -p3<br>
> ./build/l2fwd -cf -n3 -- -pf<br>
><br>
> But both cases, the aggregated performance are not
scale.<br>
><br>
> == experiments environments ==<br>
> - Two Intel 82599 NICs (total 4 ports)<br>
> - Intel Xeon X5690 @ 3.47GHz * 2 (total 12 cores)<br>
> - 1024 * 2MB hugepages<br>
> - Linux 2.6.38-15-server<br>
> - Each ports receiving 10Gbps of traffic of 64 bytes
packets, 14.88Mpps.<br>
><br>
</div>
> *1. Packet forwarding performance*<br>
<div>
<div class="h5">><br>
> In 2 ports case, receive performance is 13Mpps,<br>
> In 4 ports case, not 26Mbps, only 16.8Mpps.<br>
><br>
> Port statistics
====================================<br>
> Statistics for port 0
------------------------------<br>
> Packets sent: 4292256<br>
> Packets received: 6517396<br>
> Packets dropped: 2224776<br>
> Statistics for port 1
------------------------------<br>
> Packets sent: 4291840<br>
> Packets received: 6517044<br>
> Packets dropped: 2225556<br>
> Aggregate statistics
===============================<br>
> Total packets sent: 8584128<br>
> Total packets received: 13034472<br>
> Total packets dropped: 4450332<br>
>
====================================================<br>
><br>
> Port statistics
====================================<br>
> Statistics for port 0
------------------------------<br>
> Packets sent: 1784064<br>
> Packets received: 2632700<br>
> Packets dropped: 848128<br>
> Statistics for port 1
------------------------------<br>
> Packets sent: 1784104<br>
> Packets received: 2632196<br>
> Packets dropped: 848596<br>
> Statistics for port 2
------------------------------<br>
> Packets sent: 3587616<br>
> Packets received: 5816344<br>
> Packets dropped: 2200176<br>
> Statistics for port 3
------------------------------<br>
> Packets sent: 3587712<br>
> Packets received: 5787848<br>
> Packets dropped: 2228684<br>
> Aggregate statistics
===============================<br>
> Total packets sent: 10743560<br>
> Total packets received: 16869152<br>
> Total packets dropped: 6125608<br>
>
====================================================<br>
><br>
</div>
</div>
> *2. Packet receiving performance*<br>
<div class="HOEnZb">
<div class="h5">> I fix the codes for only receiving
packets (not forwarding),<br>
> the performance is still not scalable as each
13.3Mpps, 18Mpps.<br>
><br>
> Port statistics
====================================<br>
> Statistics for port 0
------------------------------<br>
> Packets sent: 0<br>
> Packets received: 6678860<br>
> Packets dropped: 0<br>
> Statistics for port 1
------------------------------<br>
> Packets sent: 0<br>
> Packets received: 6646120<br>
> Packets dropped: 0<br>
> Aggregate statistics
===============================<br>
> Total packets sent: 0<br>
> Total packets received: 13325012<br>
> Total packets dropped: 0<br>
>
====================================================<br>
><br>
> Port statistics
====================================<br>
> Statistics for port 0
------------------------------<br>
> Packets sent: 0<br>
> Packets received: 3129624<br>
> Packets dropped: 0<br>
> Statistics for port 1
------------------------------<br>
> Packets sent: 0<br>
> Packets received: 3131292<br>
> Packets dropped: 0<br>
> Statistics for port 2
------------------------------<br>
> Packets sent: 0<br>
> Packets received: 6260908<br>
> Packets dropped: 0<br>
> Statistics for port 3
------------------------------<br>
> Packets sent: 0<br>
> Packets received: 6238764<br>
> Packets dropped: 0<br>
> Aggregate statistics
===============================<br>
> Total packets sent: 0<br>
> Total packets received: 18760640<br>
> Total packets dropped: 0<br>
>
====================================================<br>
><br>
> The question is that<br>
> 1. How I can achieve each port receiving full
14.88Mpps ?<br>
> What might be the bottleneck in current
environment?<br>
> 2. Why the performance using multiple ports is not
scale?<br>
> I guess doubling ports shows the doubling the
receiving performance,<br>
> but it shows not. I am curious about what is
limiting the packet<br>
> receivng performance.<br>
><br>
> Thanks,<br>
> Shinae<br>
<br>
</div>
</div>
<span class="HOEnZb"><font color="#888888">--<br>
SAKURA Internet Inc. / Senior Researcher<br>
Naoto MATSUMOTO <<a moz-do-not-send="true"
href="mailto:n-matsumoto@sakura.ad.jp">n-matsumoto@sakura.ad.jp</a>><br>
SAKURA Internet Research Center <<a
moz-do-not-send="true"
href="http://research.sakura.ad.jp/" target="_blank">http://research.sakura.ad.jp/</a>><br>
<br>
</font></span></blockquote>
</div>
<br>
</div>
</blockquote>
</body>
</html>