[dpdk-users] Facing issues with Interrupt mode (IGB,VFIO)

[Stephen Hemminger]

On Tue, 13 Dec 2016 14:43:41 +0530
Prem Chaitanya <prem at telaverge.com> wrote:

> Hi All,
> 
> We modified the l3fwd application to run in interrupt+poll mode. The driver
> we used was IGB with Intel I350 NIC.
> When we ran with 1 lcore and 1 queue, there were no issues. When we tried
> to run with 2 lcores and 2 Rx-queues we are not getting interrupts.
> In one of the documentation(DPDK-201.pdf), it was mentioned that IGB driver
> doesn't support interrupt mode with multiple lcores since it supports only
> 1 epoll-fd.
> Can you please confirm if this is the case?

igb_uio device driver can not support an interrupt per queue (MSI).


> Then we went ahead and loaded the VFIO driver and tried to run the same
> application.
> Now we get an interrupt once and after that when we go back to waiting on
> epoll, we dont receive any further interrupts.
> But when we print the stats for rx-packets we see that there are packets
> waiting in the NIC Rx queue.
> Same behavior is observed even when we run application with 1 lcore and 1
> Rx queue.

You need to do some careful coding to handle interrupt+poll mode.
Look at l3fwd-power example, and even that isn't enough.

Basically, you need to dynamically go in and out of interrupt mode
and use the rx_queue_count function to check for races.

The application must have poll loop like:

while() {
          rte_epoll_wait()

          if (rx_event) {
readmore:
		rte_eth_dev_rx_intr_disable()

		while ( (n = rte_eth_rx_queue_burst()) > 0) {
			process n packets;
		}
	
		rte_eth_dev_rx_intr_enable();
		if (rte_eth_dev_rx_queue_count() > 0) {
			// lost race 
			goto readmore;
		}
	}
}