[dpdk-dev] DCA
Bruce Richardson
bruce.richardson at intel.com
Wed Apr 22 11:10:04 CEST 2015
On Tue, Apr 21, 2015 at 10:44:54AM -0700, Matthew Hall wrote:
> On Tue, Apr 21, 2015 at 10:27:48AM +0100, Bruce Richardson wrote:
> > Can you perhaps comment on the use-case where you find this binding
> > limiting? Modern platforms have multiple NUMA nodes, but they also generally
> > have PCI slots connected to those multiple NUMA nodes also, so that you can
> > have your NIC ports similarly NUMA partitionned?
>
> Hi Bruce,
>
> I was wondering if you have tried to do this on COTS (commerical
> off-the-shelf) hardware before. What I found each time I tried it was that
> PCIe slots are not very evenly distributed across the NUMA nodes unlike what
> you'd expect.
>
I doubt I've tried it on regular commercial boards as much as you guys have,
though it does happen!
> Sometimes the PCIe lanes on CPU 0 get partly used up by Super IO or other
> integrated peripherals. Other times the motherboards give you 2 x8 when you
> needed 1 x16 or they give you a bundh of x4 when you needed x8, etc.
Point taken!
>
> It's actually pretty difficult to find the mapping, for one, and even when you
> do, even harder to get the right slots for your cards and so on. In the ixgbe
> kernel driver you'll sometimes get some cryptic debug prints when it's been
> munged and performance will suffer. But in the ixgbe PMD driver you're on your
> own mostly.
It was to try and make the NUMA mapping of PCI clearer that we added in the
printing of the NUMA node on PCI scan:
EAL: PCI device 0000:86:00.0 on NUMA socket 1
EAL: probe driver: 8086:154a rte_ixgbe_pmd
EAL: PCI memory mapped at 0x7fb452f04000
EAL: PCI memory mapped at 0x7fb453004000
Is there something more than this you feel we could do in the PMD to help with
slot identification?
/Bruce
>
> Matthew.
More information about the dev
mailing list