[dpdk-dev] [PATCH v6 0/8] Interrupt mode PMD
Liu, Yong
yong.liu at intel.com
Fri Feb 27 09:00:10 CET 2015
Tested-by: Yong Liu <yong.liu at intel.com>
- Tested Commit: 00c685634b8a43e4594e26949a6c4f1cf5b67047
- OS: Fedora20 3.15.8-200.fc20.x86_64
- GCC: gcc version 4.8.3 20140911 (Red Hat 4.8.3-7) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
- NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
- Default x86_64-native-linuxapp-gcc configuration
- Total 4 cases, 4 passed, 0 failed
- Case: interrupt pmd on PF with single queue
Description: Check interrupt pmd work with single queue
Command / instruction:
Bind ports to vfio-pci.
modprobe vfio
modprobe vfio-pci
./tools/dpdk_nic_bind.py --bind=vfio-pci 0000:08:00.0 0000:08:00.1
Start l3fwd-power with one queue per port.
l3fwd-power -c 7 -n 4 -- -p 0x3 -P --config="(0,0,1),(1,0,2)"
Send one packet to Port0 and Port1, check that thread on core1 and core2
waked up.
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port1,rxq0
Expected test result:
l3fwd-power can forward packets normally and thread on core1 and core2
will sleep when there's no packet received.
- Case: interrupt pmd on PF with multi queue
Description: Check interrupt pmd work with multiple queues
Command / instruction:
Start l3fwd-power with two queues per port.
l3fwd-power -c 1f -n 4 -- -p 0x3 \
--config="(0,0,1),(0,1,2)(1,0,3),(1,1,4)"
Send packet with increased dest IP to Port0 and Port1, check that thread
on core1,core2,core3,core4 waked up.
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port1,rxq1
L3FWD_POWER: lcore 3 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 4 is waked up from rx interrupt on port1,rxq1
Expected test result:
l3fwd-power can forward packets normally and thread on core1-core4 will
sleep when there's no packet received.
- Case: interrupt pmd on PF with max Rx queues
Description: Check interrupt pmd work with maximum queues
Command / instruction:
Start l3fwd-power with 32 queues per port.
l3fwd-power -c ffffffff -n 4 -- -p 0x3 -P --config="(0,0,0),(0,1,1),\
(0,2,2),(0,3,3),(0,4,4),(0,5,5),(0,6,6),(0,7,7),(0,8,8),
(0,9,9),(0,10,10),(0,11,11),(0,12,12),(0,13,13),(0,14,14),\
(0,15,15),\
(1,0,16),(1,1,17),(1,2,18),(1,3,19),(1,4,20),(1,5,21),(1,6,22),\
(1,7,23),(1,8,24),(1,9,25),(1,10,26),(1,11,27),(1,12,28),\
(1,13,29),(1,14,30),\(1,15,31)"
Send packet with increased dest IP to Port0 and Port1, check that all
threads waked up.
Expected test result:
l3fwd-power can forward packets normally and thread on core1-core31
will sleep when there's no packet received.
- Case: interrupt pmd on VF with single queue
Description: Check interrupt pmd work on VF device
Command / instruction:
Bind ports to back to ixgbe driver.
./tools/dpdk_nic_bind.py --bind=ixgbe 0000:08:00.0 0000:08:00.1
Create one VF per Port in host and make sure PF interface up
echo 1 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs
echo 1 > /sys/bus/pci/devices/0000\:08\:00.1/sriov_numvfs
ifconfig p786p1 up
ifconfig p786p2 up
Bind VF device to vfio-pci.
./tools/dpdk_nic_bind.py --bind=vfio-pci 0000:08:10.0 0000:08:10.1
Start l3fwd-power on host with one queue per port.
l3fwd-power -c 1f -n 4 -- -p 0x3 -P --config="(0,0,1),(1,0,2)"
Send one packet to Port0 and Port1, check that thread on core1 and core2
waked up.
L3FWD_POWER: lcore 1 is waked up from rx interrupt on port1,rxq0
L3FWD_POWER: lcore 2 is waked up from rx interrupt on port1,rxq0
Expected test result:
l3fwd-power can forward packets normally on VF and thread on core1 and
core2 will sleep when there's no packet received.
> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Cunming Liang
> Sent: Friday, February 27, 2015 12:56 PM
> To: dev at dpdk.org
> Subject: [dpdk-dev] [PATCH v6 0/8] Interrupt mode PMD
>
> v6 changes
> - split rte_intr_wait_rx_pkt into two APIs 'wait' and 'set'.
> - rewrite rte_intr_rx_wait/rte_intr_rx_set.
> - using vector number instead of queue_id as interrupt API params.
> - patch reorder and split.
>
> v5 changes
> - Rebase the patchset onto the HEAD
> - Isolate ethdev from EAL for new-added wait-for-rx interrupt function
> - Export wait-for-rx interrupt function for shared libraries
> - Split-off a new patch file for changed struct rte_intr_handle that
> other patches depend on, to avoid breaking git bisect
> - Change sample applicaiton to accomodate EAL function spec change
> accordingly
>
> v4 changes
> - Export interrupt enable/disable functions for shared libraries
> - Adjust position of new-added structure fields and functions to
> avoid breaking ABI
>
> v3 changes
> - Add return value for interrupt enable/disable functions
> - Move spinlok from PMD to L3fwd-power
> - Remove unnecessary variables in e1000_mac_info
> - Fix miscelleous review comments
>
> v2 changes
> - Fix compilation issue in Makefile for missed header file.
> - Consolidate internal and community review comments of v1 patch set.
>
> The patch series introduce low-latency one-shot rx interrupt into DPDK
> with
> polling and interrupt mode switch control example.
>
> DPDK userspace interrupt notification and handling mechanism is based on
> UIO
> with below limitation:
> 1) It is designed to handle LSC interrupt only with inefficient suspended
> pthread wakeup procedure (e.g. UIO wakes up LSC interrupt handling
> thread
> which then wakes up DPDK polling thread). In this way, it introduces
> non-deterministic wakeup latency for DPDK polling thread as well as
> packet
> latency if it is used to handle Rx interrupt.
> 2) UIO only supports a single interrupt vector which has to been shared by
> LSC interrupt and interrupts assigned to dedicated rx queues.
>
> This patchset includes below features:
> 1) Enable one-shot rx queue interrupt in ixgbe PMD(PF & VF) and igb PMD(PF
> only).
> 2) Build on top of the VFIO mechanism instead of UIO, so it could support
> up to 64 interrupt vectors for rx queue interrupts.
> 3) Have 1 DPDK polling thread handle per Rx queue interrupt with a
> dedicated
> VFIO eventfd, which eliminates non-deterministic pthread wakeup latency
> in
> user space.
> 4) Demonstrate interrupts control APIs and userspace NAIP-like
> polling/interrupt
> switch algorithms in L3fwd-power example.
>
> Known limitations:
> 1) It does not work for UIO due to a single interrupt eventfd shared by
> LSC
> and rx queue interrupt handlers causes a mess.
> 2) LSC interrupt is not supported by VF driver, so it is by default
> disabled
> in L3fwd-power now. Feel free to turn in on if you want to support both
> LSC
> and rx queue interrupts on a PF.
>
> Cunming Liang (5):
> eal: declare new interrupt api
> eal/linux: add rx queue interrupt FDs to intr handle struct
> eal/bsd: dummy for new intr definition
> eal/linux: add per rx queue interrupt handling based on VFIO
> ethdev: add rx interrupt enable/disable functions
>
> Zhou, Danny (3):
> ixgbe: enable rx queue interrupts for both PF and VF
> igb: enable rx queue interrupts for PF
> l3fwd-power: enable one-shot rx interrupt and polling/interrupt mode
> switch
>
> examples/l3fwd-power/main.c | 194 ++++++++---
> lib/librte_eal/bsdapp/eal/eal_interrupts.c | 15 +
> .../bsdapp/eal/include/exec-env/rte_interrupts.h | 4 +
> lib/librte_eal/bsdapp/eal/rte_eal_version.map | 2 +
> lib/librte_eal/common/include/rte_interrupts.h | 38 +++
> lib/librte_eal/linuxapp/eal/eal_interrupts.c | 224 +++++++++---
> lib/librte_eal/linuxapp/eal/eal_pci_vfio.c | 23 +-
> .../linuxapp/eal/include/exec-env/rte_interrupts.h | 9 +
> lib/librte_eal/linuxapp/eal/rte_eal_version.map | 2 +
> lib/librte_ether/rte_ethdev.c | 66 ++++
> lib/librte_ether/rte_ethdev.h | 77 +++++
> lib/librte_ether/rte_ether_version.map | 3 +
> lib/librte_pmd_e1000/e1000_ethdev.h | 3 +
> lib/librte_pmd_e1000/igb_ethdev.c | 231 +++++++++++--
> lib/librte_pmd_ixgbe/ixgbe_ethdev.c | 377
> ++++++++++++++++++++-
> lib/librte_pmd_ixgbe/ixgbe_ethdev.h | 7 +
> 16 files changed, 1156 insertions(+), 119 deletions(-)
>
> --
> 1.8.1.4
More information about the dev
mailing list