<div dir="ltr"><div><div><div>Thanks for sharing Naoto.<br><br></div>So in your experiments, the forwarding performance still does not reach the line rate.<br></div><br>Your perf record shows that the cpu spend most of the time in polling for receiving packets, <br>
</div><div>and no other heavy operation.<br></div><div>Even though the application polling packets in its best,<br>the forwarder miss some packets from elsewhere from the application-side.<br><br></div><div>The dpdk document shows that 160Mpps forwarding performance in 2 sockets,<br>
</div><div>but I can only reach the 13 Mpps in 2 ports. <br>Even doubling the number of ports to 4 ports, the performance is still less than 17Mpps.<br><br></div><div>I want to know where is the bottleneck lies in my environments, or<br>
</div><div>how I can reprocuce the same performance as the dpdk published.<br><br></div><div>Thank you,<br></div><div>Shinae<br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, May 28, 2013 at 11:30 AM, Naoto MATSUMOTO <span dir="ltr"><<a href="mailto:n-matsumoto@sakura.ad.jp" target="_blank">n-matsumoto@sakura.ad.jp</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
FYI: Disruptive IP Networking with Intel DPDK on Linux<br>
<a href="http://slidesha.re/SeVFZo" target="_blank">http://slidesha.re/SeVFZo</a><br>
<div class="im"><br>
<br>
On Tue, 28 May 2013 11:26:30 +0900<br>
Shinae Woo <<a href="mailto:shinae2012@gmail.com">shinae2012@gmail.com</a>> wrote:<br>
<br>
> Hello, all.<br>
><br>
> I play the dpdk-1.2.3r1 with examples.<br>
><br>
> But I can not achieve the line-rate packet receive performance,<br>
> and the performance is not scale with multiple ports.<br>
><br>
> For example, in example l2fwd, I have tested two cases with 2 ports, and 4<br>
> ports,<br>
> using belowed command line each<br>
><br>
> ./build/l2fwd -cf -n3 -- -p3<br>
> ./build/l2fwd -cf -n3 -- -pf<br>
><br>
> But both cases, the aggregated performance are not scale.<br>
><br>
> == experiments environments ==<br>
> - Two Intel 82599 NICs (total 4 ports)<br>
> - Intel Xeon X5690  @ 3.47GHz * 2 (total 12 cores)<br>
> - 1024 * 2MB hugepages<br>
> - Linux 2.6.38-15-server<br>
> - Each ports receiving 10Gbps of traffic of 64 bytes packets, 14.88Mpps.<br>
><br>
</div>> *1. Packet forwarding performance*<br>
<div><div class="h5">><br>
> In 2 ports case,  receive performance is 13Mpps,<br>
> In 4 ports case,  not 26Mbps, only 16.8Mpps.<br>
><br>
> Port statistics ====================================<br>
> Statistics for port 0 ------------------------------<br>
> Packets sent:                  4292256<br>
> Packets received:              6517396<br>
> Packets dropped:               2224776<br>
> Statistics for port 1 ------------------------------<br>
> Packets sent:                  4291840<br>
> Packets received:              6517044<br>
> Packets dropped:               2225556<br>
> Aggregate statistics ===============================<br>
> Total packets sent:            8584128<br>
> Total packets received:       13034472<br>
> Total packets dropped:         4450332<br>
> ====================================================<br>
><br>
> Port statistics ====================================<br>
> Statistics for port 0 ------------------------------<br>
> Packets sent:                  1784064<br>
> Packets received:              2632700<br>
> Packets dropped:                848128<br>
> Statistics for port 1 ------------------------------<br>
> Packets sent:                  1784104<br>
> Packets received:              2632196<br>
> Packets dropped:                848596<br>
> Statistics for port 2 ------------------------------<br>
> Packets sent:                  3587616<br>
> Packets received:              5816344<br>
> Packets dropped:               2200176<br>
> Statistics for port 3 ------------------------------<br>
> Packets sent:                  3587712<br>
> Packets received:              5787848<br>
> Packets dropped:               2228684<br>
> Aggregate statistics ===============================<br>
> Total packets sent:           10743560<br>
> Total packets received:       16869152<br>
> Total packets dropped:         6125608<br>
> ====================================================<br>
><br>
</div></div>> *2. Packet receiving performance*<br>
<div class="HOEnZb"><div class="h5">> I fix the codes for only receiving packets (not forwarding),<br>
> the performance is still not scalable as each 13.3Mpps, 18Mpps.<br>
><br>
> Port statistics ====================================<br>
> Statistics for port 0 ------------------------------<br>
> Packets sent:                        0<br>
> Packets received:              6678860<br>
> Packets dropped:                     0<br>
> Statistics for port 1 ------------------------------<br>
> Packets sent:                        0<br>
> Packets received:              6646120<br>
> Packets dropped:                     0<br>
> Aggregate statistics ===============================<br>
> Total packets sent:                  0<br>
> Total packets received:       13325012<br>
> Total packets dropped:               0<br>
> ====================================================<br>
><br>
> Port statistics ====================================<br>
> Statistics for port 0 ------------------------------<br>
> Packets sent:                        0<br>
> Packets received:              3129624<br>
> Packets dropped:                     0<br>
> Statistics for port 1 ------------------------------<br>
> Packets sent:                        0<br>
> Packets received:              3131292<br>
> Packets dropped:                     0<br>
> Statistics for port 2 ------------------------------<br>
> Packets sent:                        0<br>
> Packets received:              6260908<br>
> Packets dropped:                     0<br>
> Statistics for port 3 ------------------------------<br>
> Packets sent:                        0<br>
> Packets received:              6238764<br>
> Packets dropped:                     0<br>
> Aggregate statistics ===============================<br>
> Total packets sent:                  0<br>
> Total packets received:       18760640<br>
> Total packets dropped:               0<br>
> ====================================================<br>
><br>
> The question is that<br>
> 1. How I can achieve each port receiving full 14.88Mpps ?<br>
>     What might be the bottleneck in current environment?<br>
> 2. Why the performance using multiple ports is not scale?<br>
>     I guess doubling ports shows the doubling the receiving performance,<br>
>     but it shows not. I am curious about what is limiting the packet<br>
> receivng performance.<br>
><br>
> Thanks,<br>
> Shinae<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
SAKURA Internet Inc. / Senior Researcher<br>
Naoto MATSUMOTO <<a href="mailto:n-matsumoto@sakura.ad.jp">n-matsumoto@sakura.ad.jp</a>><br>
SAKURA Internet Research Center <<a href="http://research.sakura.ad.jp/" target="_blank">http://research.sakura.ad.jp/</a>><br>
<br>
</font></span></blockquote></div><br></div>