[dpdk-dev] Debugging SR-IOV related packet flow problem with DPDK

[Andrew Harvey (agh)]

Please type the following:

dmesg | grep DMAR.  I think you will find that rebinding the uio driver
has caused issues, uio know nothing about the iommu.  If you can you may
have better luck with the vfio driver.

HTH ― Andy

On 6/17/15, 3:52 AM, "dev on behalf of Anjali Kulkarni"
<dev-bounces at dpdk.org on behalf of anjali at juniper.net> wrote:

>Folks,
>
>I am seeing this in the dmesg (VF 0 Hung DMA)
>
>[586307.789098] VF 0 setting Multicast Promiscuos Enable TRUE
>[586307.789099] Register before write : 0x9000000
>[586307.789100] Register after write : 0x19000000
>[586307.802585] dmar: DRHD: handling fault status reg 502
>[586307.802597] dmar: DMAR:[DMA Read] Request device [06:10.0] fault addr
>7c820000
>
>
>[563607.533814] DMAR:[fault reason 02] Present bit in context entry is
>clear
>[563607.544246] ixgbe 0000:06:00.1 int3: VF 0 Hung DMA
>[563608.344639] docker0: port 1(veth261ed02) entered forwarding state
>[563711.695571] VF 0 setting Multicast Promiscuos Enable FALSE
>[563711.695578] Register before write : 0x19000000
>[563711.695581] Register after write : 0x9000000
>[563711.695752] VF 0 setting Multicast Promiscuos Enable FALSE
>[563711.695758] Register before write : 0x19000000
>[563711.695761] Register after write : 0x9000000
>
>I also see some bootp and ipv6 traffic when I do tcpdump on the PF NIC.
>
>I am thinking perhaps DPDK has not completely taken over the nIC, and
>perhaps the linux driver is also somehow interacting?
>
>For intents, my DPDK app is running on the host (actually inside docker)
>and NOT inside a VM.
>
>Anjali
>
>On 6/16/15, 2:08 PM, "Anjali Kulkarni" <anjali at juniper.net> wrote:
>
>>Thanks for your response.
>>I am running my own application, and it is actually running inside a
>>docker, not a VM.
>>No VLAN configuration. Just SR-IOV is enabled on 2 NICs, and both devices
>>(/dev/uio0 and /dev/uio1) are assigned to the docker.
>>This worked a while ago perfectly.
>>What I see is that an ixia connected to the 2 10G NICs is pumping
>>traffic,
>>but the VF statistics are 0. It is as if the VF or NIC did not see the
>>packets. 
>>The source dst MAC is set correctly to the VF¹s MAC. I don¹t see any
>>prints on stdout after enabling the _DEBUG logs in config.
>>
>>Anjali
>>
>>On 6/16/15, 1:52 PM, "Thomas Monjalon" <thomas.monjalon at 6wind.com> wrote:
>>
>>>Hi,
>>>
>>>That's third email in 8 hours for this question.
>>>Do you plan to send an email every 4 hours?
>>>
>>>It's really hard to help you without knowing which application you run?
>>>in which VM? which VLAN configuration? etc
>>>
>>>2015-06-16 20:22, Anjali Kulkarni:
>>>> Hi,
>>>> 
>>>> Can someone please help with this? How can packet flow be traced on
>>>>DPDK?
>>>> Where are all the logs stored if we do enable the _DEBUG configs in
>>>> config/ directory?
>>>
>>>stdout or somewhere else if you configured logs in your app.
>>>
>>>> Anjali
>>>[...]
>>>> I am observing that the SR-IOV enabled NIC does not increment any VF
>>>>packet
>>>> counters despite incoming packet flow. I have enabled some debug logs
>>>>like
>>>> CONFIG_RTE_LIBRTE_IXGBE_DEBUG_RX in config file.
>>>> Can someone point me to where these logs are being printed so I can
>>>>try
>>>>to
>>>> figure out where packets are being dropped? Or how can I debug DPDK
>>>>packet
>>>> flow related issues (are there any stats I can look at)?
>>>
>>>Statistics can be requested through a dedicated ethdev API.
>>>
>>
>