<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div style="-webkit-text-size-adjust: auto; "><br></div><div style="-webkit-text-size-adjust: auto; ">On May 24, 2013, at 7:41 AM, Thomas Monjalon <<a href="mailto:thomas.monjalon@6wind.com">thomas.monjalon@6wind.com</a>> wrote:<br><br></div><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><span>Hello,</span><br><span></span><br><span>24/05/2013 16:11, Patrick Mahan :</span><br><blockquote type="cite"><span>Intel Xeon E5-2690 (8 physical, 16 virtual)</span><br></blockquote><span></span><br><span>How many CPU sockets have you ?</span><br></div></blockquote><div style="-webkit-text-size-adjust: auto; "><br></div><span style="-webkit-text-size-adjust: auto;">This is a Dell PowerEdge T620, it has two sockets, but only one has a CPU in it.</span><div><br><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><span></span><br><blockquote type="cite"><span>64 Gbyte DDR3 memory</span><br></blockquote><blockquote type="cite"><span>Intel 82599EB-SPF dual port 10GE interface</span><br></blockquote><blockquote type="cite"><span>CentOS 6.4 (2.6.32-358.6.1.el6.x86_64)</span><br></blockquote><blockquote type="cite"><span>The 82599 is in a 16x PCI-e slot.</span><br></blockquote><span></span><br><span>Check the datasheet of your motherboard.</span><br><span>Are you sure it is wired as a 16x PCI-e ?</span><br></div></blockquote><div style="-webkit-text-size-adjust: auto; "><br></div><span style="-webkit-text-size-adjust: auto;">As far as I can tell from the specs on the Dell site - </span><span style="-webkit-text-size-adjust: auto; background-color: rgba(255, 255, 255, 0);">www.<b>dell</b>.com/us/business/p/<b>poweredge</b>-<b>t620</b>/pd</span></div><div><span style="-webkit-text-size-adjust: auto;"><br></span><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><span>Is it connected to the right NUMA node ?</span><br></div></blockquote><div><br></div>Yes, it's in slot labeled: PCIE_G3_x16 (cpu1). The interfaces show up as p2p1 and p2p2.</div><div><br><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><span></span><br><blockquote type="cite"><span>I have it attached to an IXIA box. I have been running the app 'testpmd'</span><br></blockquote><blockquote type="cite"><span>in iofwd mode with 2K rx/tx descriptors and 512 burst/mbcache. I have been</span><br></blockquote><blockquote type="cite"><span>varying the # of queues and unfortunately, I am not seeing full line rate.</span><br></blockquote><span></span><br><span>What is your command line ?</span><br><span></span><br></div></blockquote><div><br></div>sudo build/app/testpmd -b 0000:03:00.0 -b 0000:03:00.1 -c<coremask> -n3 -- --nb-cores=<ncores> --nb-ports=2 --rxd=2048 --rxd=2048 --mbcache=512 --burst=512 --rxd=<nqueues> --txq=<nqueues></div><div><br></div><div>Where I am using the following to determine cores, coremask and nqueues:</div><div><br></div><div> ncores = nqueues * 2 // this actually the number of ports being tested</div><div> coremask = (1 << (ncores + 1)) - 1</div><div><br></div><div>So for say, 3 rx/tx queues -</div><div><br></div><div> ncores = 3 * 2 = 6</div><div> coremask = (1 << (6 + 1)) - 1 = 127 (0x7f)</div><div><br></div><div>Now that I remember it I had to fix testpmd to allocate enough mbufs.</div><div><br></div><div><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><blockquote type="cite"><span>I am seeing about 20-24% droppage on the receive side. It doesn't seem to</span><br></blockquote><blockquote type="cite"><span>matter the # of queues.</span><br></blockquote><span></span><br><span>If queues are polled by different cores, it should matter.</span><br></div></blockquote><div><br></div>I assume you mean different physical cores, yes?</div><div><br></div><div>There is only one physical core, but each 'forwarding' thread is on a separate core.</div><div><br><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><span></span><br><blockquote type="cite"><span>Question 1: Is 'testpmd' the best application for this type of testing? If</span><br></blockquote><blockquote type="cite"><span>not, which program? Or do I need to roll my own?</span><br></blockquote><span></span><br><span>testpmd is the right application for performance benchmark.</span><br><span>It is also possible to use examples l2fwd/l3fwd but you should keep testpmd.</span><br></div></blockquote><div><br></div>I am just starting with testpmd to get a feel for raw throughput. I want to test l2 and l3 soon but I may loose access to the Ixia.</div><div><br></div><div><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><span></span><br><blockquote type="cite"><span>Question 2: I have blacklisted the Intel i350 ports on the motherboard and</span><br></blockquote><blockquote type="cite"><span>am using ssh to access the platform. Could this be affecting the test?</span><br></blockquote><span></span><br><span>You mean i350 is used for ssh ? It shouldn't significantly affect your test.</span><br></div></blockquote><div><br></div>Okay, I noticed that they scanned by the pci dpdk layer.</div><div><br></div><div>Thanks,</div><div><br></div><div>Patrick</div><div><br><blockquote type="cite" style="-webkit-text-size-adjust: auto; "><div><span></span><br><span>-- </span><br><span>Thomas</span><br></div></blockquote></div></body></html>