[dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a bus
Wiles, Keith
keith.wiles at intel.com
Wed Feb 15 19:09:06 CET 2017
> On Feb 15, 2017, at 11:25 AM, Jerin Jacob <jerin.jacob at caviumnetworks.com> wrote:
>
> On Wed, Feb 15, 2017 at 02:27:47PM +0000, Shreyansh Jain wrote:
>>> -----Original Message-----
>>> From: Wiles, Keith [mailto:keith.wiles at intel.com]
>>> Sent: Wednesday, February 15, 2017 7:53 PM
>>> To: Shreyansh Jain <shreyansh.jain at nxp.com>
>>> Cc: Jan Blunck <jblunck at infradead.org>; dev at dpdk.org
>>> Subject: Re: [dpdk-dev] [PATCH 3/7] eal: move virtual device probing into a
>>> bus
>>>
>>>
>>>> On Feb 15, 2017, at 8:15 AM, Shreyansh Jain <shreyansh.jain at nxp.com> wrote:
>>>>
>>>> On Wednesday 15 February 2017 07:41 PM, Shreyansh Jain wrote:
>>>>> On Wednesday 15 February 2017 03:32 PM, Jan Blunck wrote:
>>>>>> This is a refactoring of the virtual device probing which moves into into
>>>>>> a proper bus structure.
>>>>>>
>>>>>> Signed-off-by: Jan Blunck <jblunck at infradead.org>
>>>>>> ---
>>>>>> lib/librte_eal/common/eal_common_dev.c | 22 -----------------
>>>>>> lib/librte_eal/common/eal_common_vdev.c | 44
>>>>>> +++++++++++++++++++++++++++++++++
>>>>>> 2 files changed, 44 insertions(+), 22 deletions(-)
>>>>>>
>>>>>
>>>>> [...]
>>>>>
>>>>>>
>>>>>> diff --git a/lib/librte_eal/common/eal_common_vdev.c
>>>>>> b/lib/librte_eal/common/eal_common_vdev.c
>>>>>> index 7d6e54f..523a3d6 100644
>>>>>> --- a/lib/librte_eal/common/eal_common_vdev.c
>>>>>> +++ b/lib/librte_eal/common/eal_common_vdev.c
>>>>>> @@ -37,8 +37,10 @@
>>>>>> #include <stdint.h>
>>>>>> #include <sys/queue.h>
>>>>>>
>>>>> [...]
>>>>>
>>>>>> +
>>>>>> +static struct rte_bus rte_vdev_bus = {
>>>>>> + .scan = vdev_scan,
>>>>>> + .probe = vdev_probe,
>>>>>> +};
>>>>>> +
>>>>>> +RTE_REGISTER_BUS_LATE(virtual, rte_vdev_bus);
>>>>>>
>>>>>
>>>>> Does it matter if VDEV buses are registered before or after other
>>>>> buses? Either way, the callbacks would be called in the order specified
>>>>> in EAL.
>>>>>
>>>>>
>>>>
>>>> Just ignore this comment - I am misunderstood something.
>>>>
>>>> But another question: Is there specific reason VDEV should be
>>> registered/scanned *after* other devices? Is there some specific problem if
>>> we do otherwise? (I think this is should be done, but I don't have a specific
>>> reason).
>>>
>>> Does the bonding driver which uses physical devices need to be registered
>>> after physical ones? In Pktgen I noticed the vdev after the physical ports
>>> and I could not blacklist them as the bonding driver needed them, which
>>> caused the bonding ports to have a greater port number. In the case of pktgen
>>> the bonding ports were up around 8 or 10 and caused the display to not show
>>> the bonding ports. This is really just a usability problem for the developer
>>> using Pktgen. I would like to see the vdev devices first, but as long as the
>>> drivers (like bonding) are fine with them being first.
>>
>> Ah, now I remember - there was a patch from Jerin for this.
>> Probably he is the best person to comment here.
>> (I don't have much insight here).
>
> commit f4ce209a8ce5f416b61c76cee773bc54749e2048
> Author: Jerin Jacob <jerin.jacob at caviumnetworks.com>
> Date: Sun Nov 20 13:30:50 2016 +0530
>
> eal: postpone vdev initialization
>
> Some platform like octeontx may use pci and
> vdev based combined device to represent a logical
> dpdk functional device.In such case, postponing the
> vdev initialization after pci device
> initialization will provide the better view of
> the pci device resources in the system in
> vdev's probe function, and it allows better
> functional subsystem registration in vdev probe
> function.
>
> As a bonus, This patch fixes a bond device
> initialization use case.
>
> example command to reproduce the issue:
> ./testpmd -c 0x2 --vdev 'eth_bond0,mode=0,
> slave=0000:02:00.0,slave=0000:03:00.0' --
> --port-topology=chained
>
> root cause:
> In existing case(vdev initialization and then pci
> initialization), creates three Ethernet ports with
> following port ids
> 0 - Bond device
> 1 - PCI device 0
> 2 - PCI devive 1
>
> Since testpmd, calls the configure/start on all the ports on
> start up,it will translate to following illegal setup sequence
I guess I see this differently, meaning we modified the system to put vdev devices last only because we do not have clean way to startup the system for pdev/vdev devices. The application should be agnostic to the devices being started and the system needs to determine the correct order without a chicken and egg problem. The test-pmd application just starts from 0 to n to initialize devices, which he should be able to do in any order. It is possible the application could initialize the devices (pdev/vdev) in any order, which the current design would break if they tried to init the bonding driver first.
What happens if a vdev needs to be initialized before a pdev device?
Not saying we need to solve this problem now, but need to figure this out some how. Maybe we need a priority for pdev/vdev devices to determine init order????
>
> 1)bond device configure/start
> 1.1) pci device0 stop/configure/start
> 1.2) pci device1 stop/configure/start
> 2)pci device 0 configure(illegal setup case,
> as device in start state)
>
> The fix changes the initialization sequence and
> allow initialization in following valid setup order
> 1) pcie device 0 configure/start
> 2) pcie device 1 configure/start
> 3) bond device 2 configure/start
> 3.1) pcie device 0/stop/configure/start
> 3.2) pcie device 1/stop/configure/start
Regards,
Keith
More information about the dev
mailing list