[dpdk-users] [ovs-dev] adding dpdk ports sharing same pci address to ovs-dpdk bridge

[devendra rawat]

Hi Ciara,

On Fri, Oct 6, 2017 at 7:30 PM, Loftus, Ciara <ciara.loftus at intel.com>
wrote:

> >
> > On Thu, Sep 21, 2017 at 1:58 PM, Loftus, Ciara <ciara.loftus at intel.com>
> > wrote:
> > > 21/09/2017 10:04, Loftus, Ciara:
> > > > > 20/09/2017 19:33, Kevin Traynor:
> > > > > > On 09/08/2017 10:56 AM, Loftus, Ciara wrote:
> > > > > > > It seems the DPDK function rte_eth_dev_get_port_by_name() will
> > > > > > > always return the port ID of the first port on your NIC,
> > > > > > > when you specify the single PCI address and that's where the
> > > > > > > problem is. There doesn't seem to be a way currently to
> > > > > > > indicate to the calling application that in fact two
> > > > > > > (or more) port IDs are associated with the one PCI address.
> > > > >
> > > > > We have two ports (with the same PCI address) so we should have
> > > > > two different names.
> > > > > Where the names passed to rte_eth_dev_get_port_by_name() come
> > > from?
> > > > > It is the user parameter from options:dpdk-devargs=0002:01:00.0,
> > right?
> > > >
> > > > Yes, we're using the PCI address specified by the user in
> dpdk-devargs.
> > > >
> > > > > > > I am cc-ing DPDK users mailing list for hopefully some input.
> > > > > > > Are there any plans for the rte_eth_dev_get_port_by_name
> > > function
> > > > > > > to be compatible with NICs with multiple ports under the same
> PCI
> > > address?
> > > > >
> > > > > We cannot return two different ports for the same name.
> > > > > There are two issues here:
> > > > >   - the input should not be the PCI address
> > > > >   - the ethdev function should look at ethdev name, not rte_device
> > > > > one
> > > >
> > > > This would require the user having to "guess" the DPDK ethdev name
> > > > which is something we'd like to avoid.
> > >
> > > Yes, but you can provide a way to list the ports with their names
> > > and characteristics.
> > Ok, I see. Maybe something like this could be considered:
> >
> > port A = dpdk-devargs=xx:yy:zz 0
> > port B = dpdk-devargs=xx:yy:zz 1
> >
> > If we detect a value after the PCI address we iterate through the
> > rte_eth_dev_info
> > (http://dpdk.org/doc/api/structrte__eth__dev__info.html) for all valid
> port
> > IDs and assign port A to the first ethdev encountered with the provided
> PCI
> > address, and port B to the second, etc.
> >
> > If we don't detect a value, then we operate as normal. Thoughts?
> >
> > Hi Everyone,
> >
> > Anything finalized for sorting out this issue, do you need any more
> > information regarding this issue ?
>
> Hi,
>
> I put together a very rough RFC that aims to work-around the issue:
> https://mail.openvswitch.org/pipermail/ovs-dev/2017-October/339496.html
>
> It hasn't been tested as I don't have access to the type of card that has
> one PCI for multiple ports.
> If anybody does have access to such a device, I welcome you to try the
> patch although I'm not hopeful it will succeed first -pass.
> Looking for feedback on implementation, interface, etc.
>
>  I applied the patch on top of OVS v2.8.1 and used DPDK v17.08. I used
Mellanox ConnectX-3 pro NIC for testing, this NIC provides two 10G ports
that share a single PCI address.

The patch is working fine, I was able to add both the 10G ports to OVS
bridge by specifying the port no. (0 or 1) in the dpdk-devargs.

# ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
options:dpdk-devargs=0002:01:00.0,0
# ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
options:dpdk-devargs=0002:01:00.0,1

the port no. (0 and 1) are bound to actual physical port, i.e if I keep on
adding and deleting port no. 0 to bridge br0 multiple times,
every time I add the port back to br0, the same physical port is added to
bridge br0.

Thanks,
Devendra


> >
> > Thanks,
> > Devendra
> >
> >
> > >
> > > > We had the same problem using DPDK port IDs and decided not to use
> > > > them anymore, and use the PCI instead as it took the guesswork out.
> > > > Ethdev names and port IDs can change between tests, unlike the PCI
> > > > address which tends to remain constant for a device.
> > >
> > > We can add a requirement on ethdev names and make sure they remain
> > > constant for a given port.
> > >
> > > > > The idea is that we have only one rte_device object and we
> instantiate
> > > > > two rte_eth_dev ports.
> > > > > An ethdev port can be identified with its id (a number) or its
> unique
> > > name.
> > > > > Unfortunately, the user cannot guess the port id or the name set
> by the
> > > > > PMD.
> > > >
> > > > Exactly. Thanks for clarifying what's going on under the hood.
> > > >
> > > > Ciara
> > > >
> > > > >
> > > > > > Hi Adrien/Nelio,
> > > > > >
> > > > > > Is this something you can answer? We're wondering how to handle
> > this
> > > in
> > > > > > OVS and whether a temporary or long term solution is needed.
> > > > >
> > > > > I suggest to rely on ethdev name.
> > > > > You will need to show to the user the mapping between the bus
> > > information
> > > > > (PCI id here) and the device names.
> > > > >
> > > > > Another alternative is to add a new function returning all ethdev
> ports
> > > > > associated to a given rte_device resource.
> > > > > So you would get two ports and you could pick one on the first
> "add-
> > > port",
> > > > > and the other one for the second "add-port" command.
> > > > > It means the user would be forced to add them in the right order
> if he
> > > > > wants a reproducible result.
>
>