<div><a name="OLE_LINK1">QPI bandwidth is definitely large enough, but it seems that QPI is only responsible for the communication between separate CPU chips. What you need to do is actually accessing the memory on the other part, probably not even hit the bandwidth. The latency can be caused by a lot of facts during a NUMA operation.</a></div><div><div>/Bob</div><div> </div><div style="font-size: 12px;font-family: Arial Narrow;padding:2px 0 2px 0;">------------------ 原始邮件 ------------------</div><div style="font-size: 12px;background:#efefef;padding:8px;"><div>发件人: "Zachary";<zachary.jen@cas-well.com>;</div><div>发送时间: 2013年9月2日(星期一) 中午11:22</div><div>收件人: "dev"<dev@dpdk.org>; </div><div>抄送: " "Yannic.Chou (周哲正) : 6808" <yannic.chou@cas-well.com>; "Alan Yu 俞亦偉 : 6632""<Alan.Yu@cas-well.com>; </div><div>主题: [dpdk-dev] DPDK & QPI performance issue in Romley platform.</div></div><div> </div> Hi~ I have a question about DPDK & QPI performance issue in Romley platform. Recently, I use DPDK example, l2fwd, to test DPDK's performance in my Romley platform. When I try to do the test, crossing used CPU, I find the performance dramatically decrease. Is it true? Or any method can prove the phenomenon? In my opinion, there should be no this kind of issue here due to QPI have enough bandwidth to deal the kinds of case. Thus, I am so amaze in our results and can not explain it. Could someone can help me to solve this problem. Thank a lot! My testing environment describe as below: Platform: Romley CPU: E5-2643 * 2 RAM: Transcend <big>8GB PC3-1600 DDR3 * 8</big> <big>OS: Fedora core 14</big> <big>DPDK: v1.3.1r2, example/l2fwd Slot setting: SlotA is controled by CPU1 directly.</big> <big> SlotB is controled by CPU0 directly. <big> </big></big><big>DPDK pre-setting: a. BIOS setting: HT=disable b. Kernel paramaters isolcpus=2,3,6,7 default_hugepagesz=1024M hugepagesz=1024M hugepages=16 c. OS setting: service avahi-daemon stop service NetworkManager stop service iptables stop service acpid stop selinux disable Example program Command: a. SlotB(CPU0) -> CPU1 #>./l2fwd -c 0xc -n 4 -- -q 1 -p 0xc b. SlotA(CPU1) -> CPU0 #>./l2fwd -c 0xc0 -n 4 -- -q 1 -p 0xc0 Results: use frame size 128 bytes</big> <table class="MsoNormalTable" border="0" cellpadding="0" cellspacing="0" style="margin-left:20.9pt; border-collapse:collapse"> <tbody> <tr> <td valign="top" width="207" style="width:124.0pt; border:solid windowtext 1.0pt; padding:0cm 5.4pt 0cm 5.4pt"> CPU Affinity </td> <td valign="top" width="242" style="width:144.95pt; border:solid windowtext 1.0pt; border-left:none; padding:0cm 5.4pt 0cm 5.4pt"> Slot A (CPU1) </td> <td valign="top" width="242" style="width:144.95pt; border:solid windowtext 1.0pt; border-left:none; padding:0cm 5.4pt 0cm 5.4pt"> Slot B (CPU0) </td> </tr> <tr> <td valign="top" width="207" style="width:124.0pt; border:solid windowtext 1.0pt; border-top:none; padding:0cm 5.4pt 0cm 5.4pt"> CPU0 </td> <td valign="top" width="242" style="width:144.95pt; border-top:none; border-left:none; border-bottom:solid windowtext 1.0pt; border-right:solid windowtext 1.0pt; padding:0cm 5.4pt 0cm 5.4pt"> 15.9% </td> <td valign="top" width="242" style="width:144.95pt; border-top:none; border-left:none; border-bottom:solid windowtext 1.0pt; border-right:solid windowtext 1.0pt; padding:0cm 5.4pt 0cm 5.4pt"> 96.49% </td> </tr> <tr> <td valign="top" width="207" style="width:124.0pt; border:solid windowtext 1.0pt; border-top:none; padding:0cm 5.4pt 0cm 5.4pt"> CPU1 </td> <td valign="top" width="242" style="width:144.95pt; border-top:none; border-left:none; border-bottom:solid windowtext 1.0pt; border-right:solid windowtext 1.0pt; padding:0cm 5.4pt 0cm 5.4pt"> 90.88% </td> <td valign="top" width="242" style="width:144.95pt; border-top:none; border-left:none; border-bottom:solid windowtext 1.0pt; border-right:solid windowtext 1.0pt; padding:0cm 5.4pt 0cm 5.4pt"> 24.78% </td> </tr> </tbody> </table> 本信件可能包含瑞祺電通機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。 This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient.</div>