[dpdk-users] Facing issues with Interrupt mode (IGB,VFIO)

[Prem Chaitanya]

Hi Stephen,

Thanks a lot for your response.
We wrote our application(by modifying l3fwd application) and by looking at
the l3fwd-power sample application. But didnt know about this race you are
talking about.
Question: If we enable interrupt and go into epoll_wait, wont the
interrupts be pending in eventfd ? Or am i missing something.

I applied the queue_count check and tried to read again in case there are
packets. But still i see there is a lot of inconsistency in receiving
interrupts.
After i did this change, first time i ran the application i saw interrupts
coming in, but from subsequent time i don't see them anymore.

Here is the our code flow.

1. Register for interrupts
2. Poll Rx-Queue.
3. If idle_poll crosses configured limit, turn-on interrupt for that queue
and goto epoll_wait
4. One interrupt is received then disable interrupt for that queue and poll.

Let me know if you need any specific information.


On Wed, Dec 14, 2016 at 3:52 AM, Stephen Hemminger <
stephen at networkplumber.org> wrote:

> On Tue, 13 Dec 2016 14:43:41 +0530
> Prem Chaitanya <prem at telaverge.com> wrote:
>
> > Hi All,
> >
> > We modified the l3fwd application to run in interrupt+poll mode. The
> driver
> > we used was IGB with Intel I350 NIC.
> > When we ran with 1 lcore and 1 queue, there were no issues. When we tried
> > to run with 2 lcores and 2 Rx-queues we are not getting interrupts.
> > In one of the documentation(DPDK-201.pdf), it was mentioned that IGB
> driver
> > doesn't support interrupt mode with multiple lcores since it supports
> only
> > 1 epoll-fd.
> > Can you please confirm if this is the case?
>
> igb_uio device driver can not support an interrupt per queue (MSI).
>
>
> > Then we went ahead and loaded the VFIO driver and tried to run the same
> > application.
> > Now we get an interrupt once and after that when we go back to waiting on
> > epoll, we dont receive any further interrupts.
> > But when we print the stats for rx-packets we see that there are packets
> > waiting in the NIC Rx queue.
> > Same behavior is observed even when we run application with 1 lcore and 1
> > Rx queue.
>
> You need to do some careful coding to handle interrupt+poll mode.
> Look at l3fwd-power example, and even that isn't enough.
>
> Basically, you need to dynamically go in and out of interrupt mode
> and use the rx_queue_count function to check for races.
>
> The application must have poll loop like:
>
> while() {
>           rte_epoll_wait()
>
>           if (rx_event) {
> readmore:
>                 rte_eth_dev_rx_intr_disable()
>
>                 while ( (n = rte_eth_rx_queue_burst()) > 0) {
>                         process n packets;
>                 }
>
>                 rte_eth_dev_rx_intr_enable();
>                 if (rte_eth_dev_rx_queue_count() > 0) {
>                         // lost race
>                         goto readmore;
>                 }
>         }
> }
>



-- 
Regards
Prem