DPDK version tested: 19.11.1 & 20.2 section: 28.5, Flow config for VF-0 for ipv4-other Rule: 1. `flow create 0 ingress pattern eth / ipv4 src is 192.168.0.1 dst is 192.168.0.2 proto is 3 / vf id is 0 / end actions queue index 1 / end` 2. `flow create 0 ingress pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end` 3. `flow create 0 ingress pattern eth / ipv4 proto is 17 / vf id is 0 / end actions queue index 1 / end` Result: `Caught error type 13 (specific pattern item): cause: 0x17ff5b140, Matching VF traffic without affecting it (transfer attribute) is unsupported: Operation not supported` CMD: a. PF: ``` ./x86_64-native-linuxapp-gcc/app/testpmd -l 14-15 -w 08:00.0 --socket-mem=128 -- -i --rxq=4 --txq=4 --disable-rss --pkt-filter-mode=perfect``` b. VF: ```./x86_64-native-linuxapp-gcc/app/testpmd -l 16-17 -w 0000:08:02.0 --socket-mem=128,0 --file-prefix=vf -- -i --rxq=4 --txq=4 --disable-rss --pkt-filter-mode=perfect``` on both: ``` set fwd rxonly set verbose 1 start ```
Can you please check. Thanks
(In reply to Ajit Khaparde from comment #1) > Can you please check. Thanks you should download rule as this after echo vf: flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end
not a issue, it is a usage problem
(In reply to Vipin Varghese from comment #0) > DPDK version tested: 19.11.1 & 20.2 > > section: 28.5, Flow config for VF-0 for ipv4-other > > Rule: > > 1. `flow create 0 ingress pattern eth / ipv4 src is 192.168.0.1 dst is > 192.168.0.2 proto is 3 / vf id is 0 / end actions queue index 1 / end` > 2. `flow create 0 ingress pattern eth / ipv4 src is 11.11.11.11 dst is > 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end` > 3. `flow create 0 ingress pattern eth / ipv4 proto is 17 / vf id is 0 / end > actions queue index 1 / end` > hi, you should add "transfer" in your cmd for rule of vf
Hi Ajit & Wei, Reopening the ticket, based on the discussion via email. Reason: Unable to find example or use case with testpmd explaining the same. Following files reference to the verb is found 1. guides/testpmd_app_ug/testpmd_funcs.rst 2. guides/prog_guide/rte_flow.rst 3. guides/howto/vfd.rst Really appreciate if the changes can be reflect in all or selected files of 1. `https://doc.dpdk.org/dts/test_plans/ddp_mpls_test_plan.html` 2. `https://doc.dpdk.org/dts/test_plans/generic_flow_api_test_plan.html` 3. `https://doc.dpdk.org/dts/test_plans/rss_to_rte_flow_test_plan.html
Hi Wei & Ajit, reopening for DPDK 20.05-rc2
what I can help for flow download has been done, more process of testplan doc seems not my cover.
Test update: Using with proposed `transfer` on DPDK 19.11.1 LTS is not working for me. DPDK 19.11.1 LTS PF: 0000:08:00.0 VF: 0000:08:02.0 Steps Followed ``` 1. echo 2 > /sys/bus/pci/devices/0000\:08\:00.0/max_vfs 2. cat /sys/bus/pci/devices/0000\:08\:00.0/max_vfs 3. echo 1 > /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs 4. bind PF with `igb_uio` 5. bind VF with `vfio-pci` 6. start PF with `testpmd` - `./testpmd -l 10 -w 0000:08:00.0 -n 4 -- -i --forward-mode=mac --rxq=1 --txq=1` 7. Send Packets without rule, and confirm with stats 8. start VF with `l2fwd` 9. Add flow rule - `flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end ` 10. check for packets in VF, check for packet in PF ``` Test result: a) Redirect to VF did not work towards `l2fwd` b) stopping the application and bind back to i40e kernel driver lead to kernel Panic and hung state.
Wei - can you check why its not working for Vipin?
hi, this is a new problem for "l2fwd", not the old one, please find the maintainer for it.
Hi Wei I disagree with your comment `it is a l2fwd` issue. In my testing, binding with `i40e-vf` did not also work. Hence please help me sort the VF transfer issue. @Ajith: with my testing and observation in both kernel and dpdk, VF redirect did not work for me.
Have retested the scenario once again, Application: PF with testpmd, VF with testpmd Rule set: only to PF testpmd driver: igb_uio, vfio-pci driver: 2.1.14-k firmware: 6.01 result vfio: rule set error result with igb-uio: rule gets added, but I am unable to get the flow director transfer from PF to VF Summary: all packets received by PF only and not issue in the app for VF ``` testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 0 / end i40e_check_write_reg(): [0x00267318] original: 0x00000000 i40e_check_write_reg(): [0x00267318] after: 0x000dff00 i40e_check_write_reg(): [0x0026731c] original: 0x00000000 i40e_check_write_reg(): [0x0026731c] after: 0x00000000 i40e_check_write_reg(): [0x002508c0] original: 0x00000000 i40e_check_write_reg(): [0x002508c0] after: 0x00000000 i40e_check_write_reg(): [0x002508e0] original: 0x00018018 i40e_check_write_reg(): [0x002508e0] after: 0x00058018 i40e_fdir_setup(): FDIR HW Capabilities: num_filters_guaranteed = 256, num_filters_best_effort = 7168. i40e_vsi_get_bw_config(): VSI bw limit:0 i40e_vsi_get_bw_config(): VSI max_bw:0 i40e_vsi_get_bw_config(): VSI TC0:share credits 1 i40e_vsi_get_bw_config(): VSI TC0:credits 0 i40e_vsi_get_bw_config(): VSI TC0: max credits: 0 i40e_vsi_get_bw_config(): VSI TC1:share credits 0 i40e_vsi_get_bw_config(): VSI TC1:credits 0 i40e_vsi_get_bw_config(): VSI TC1: max credits: 0 i40e_vsi_get_bw_config(): VSI TC2:share credits 0 i40e_vsi_get_bw_config(): VSI TC2:credits 0 i40e_vsi_get_bw_config(): VSI TC2: max credits: 0 i40e_vsi_get_bw_config(): VSI TC3:share credits 0 i40e_vsi_get_bw_config(): VSI TC3:credits 0 i40e_vsi_get_bw_config(): VSI TC3: max credits: 0 i40e_vsi_get_bw_config(): VSI TC4:share credits 0 i40e_vsi_get_bw_config(): VSI TC4:credits 0 i40e_vsi_get_bw_config(): VSI TC4: max credits: 0 i40e_vsi_get_bw_config(): VSI TC5:share credits 0 i40e_vsi_get_bw_config(): VSI TC5:credits 0 i40e_vsi_get_bw_config(): VSI TC5: max credits: 0 i40e_vsi_get_bw_config(): VSI TC6:share credits 0 i40e_vsi_get_bw_config(): VSI TC6:credits 0 i40e_vsi_get_bw_config(): VSI TC6: max credits: 0 i40e_vsi_get_bw_config(): VSI TC7:share credits 0 i40e_vsi_get_bw_config(): VSI TC7:credits 0 i40e_vsi_get_bw_config(): VSI TC7: max credits: 0 i40e_fdir_setup(): FDIR setup successfully, with programming queue 0. i40e_fdir_rx_proc_enable(): Flow Director processing on RX set to 1 i40e_flow_fdir_filter_programming(): filling filter programming descriptor. i40e_flow_fdir_filter_programming(): filling transmit descriptor. Flow rule #0 created ``` ``` igb_uio: $ cat /sys/bus/pci/devices/0000\:08\:00.0/max_vfs 0 $ sudo vim /sys/bus/pci/devices/0000\:08\:00.0/max_vfs $ cat /sys/bus/pci/devices/0000\:08\:00.0/max_vfs 1 $ lspci -ks 0000:08:06.0 08:06.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 01) Subsystem: Intel Corporation Ethernet Virtual Function 700 Series Kernel driver in use: igb_uio Kernel modules: i40evf $ sudo ../../usertools/dpdk-devbind.py -b vfio-pci 0000:08:02.0 $ lspci -ks 0000:08:02.0 08:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 01) Subsystem: Intel Corporation Ethernet Virtual Function 700 Series Kernel driver in use: vfio-pci Kernel modules: i40evf ``` ``` vfio-pci: # lspci -ks 0000:08:00.0 08:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) Subsystem: Intel Corporation Ethernet Converged Network Adapter X710-4 Kernel modules: i40e # modprobe vfio-pci # ./usertools/dpdk-devbind.py -b vfio-pci 0000:08:00.0 # lspci -ks 0000:08:00.0 08:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) Subsystem: Intel Corporation Ethernet Converged Network Adapter X710-4 Kernel driver in use: vfio-pci Kernel modules: i40e 1# cat /sys/bus/pci/devices/0000\:08\:00.0/sriov_totalvfs 32 root@saesrv02:~/Downloads/dpdksrc/dpdk-stable-19.11.1# cat /sys/bus/pci/devices/0000\:08\:00.0/sriov_numvfs 0 "/sys/devices/pci0000:00/0000:00:03.0/0000:08:00.0/sriov_numvfs" "/sys/devices/pci0000:00/0000:00:03.0/0000:08:00.0/sriov_numvfs" E514: write error (file system full?) # cat /sys/devices/pci0000\:00/0000\:00\:03.0/0000\:08\:00.1/sriov_numvfs 0 # vim /sys/devices/pci0000\:00/0000\:00\:03.0/0000\:08\:00.1/sriov_numvfs # cat /sys/devices/pci0000\:00/0000\:00\:03.0/0000\:08\:00.1/sriov_numvfs 1 # lshw -c network -businfo Bus info Device Class Description ======================================================== pci@0000:01:00.0 enp1s0f0 network I350 Gigabit Network Connection pci@0000:01:00.1 enp1s0f1 network I350 Gigabit Network Connection pci@0000:08:00.0 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:08:00.1 ens261f1 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:08:00.2 ens261f2 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:08:00.3 ens261f3 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:08:06.0 enp8s6 network Illegal Vendor ID pci@0000:81:00.0 ens259f0 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:81:00.1 ens259f1 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:83:00.0 ens801f0 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:83:00.1 ens801f1 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:83:00.2 ens801f2 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:83:00.3 ens801f3 network Ethernet Controller X710 for 10GbE SFP+ # lspci -ks 0000:08:06.0 08:06.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 01) Subsystem: Intel Corporation Ethernet Virtual Function 700 Series Kernel driver in use: i40evf Kernel modules: i40evf # ./testpmd -l 10-11 -w 0000:08:00.0 -n 4 -- -i --forward-mode=mac --rxq=1 --txq=1 13:21:11.787610 IP 11.11.11.11.1234 > 12.12.12.12.5678: UDP, length 12 13:21:11.787679 IP 11.11.11.11.1234 > 12.12.12.12.5678: UDP, length 12 testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end Caught error type 13 (specific pattern item): cause: 0x17ff555c0, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 0 / end Caught error type 13 (specific pattern item): cause: 0x17ff555c0, Invalid VF ID for FDIR.: Invalid argument # ../../usertools/dpdk-devbind.py -b vfio-pci 0000:08:06.0 $ lspci -ks 0000:08:06.0 08:06.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 01) Subsystem: Intel Corporation Ethernet Virtual Function 700 Series Kernel driver in use: vfio-pci Kernel modules: i40evf flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end Caught error type 13 (specific pattern item): cause: 0x17ff555c0, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 0 / end Caught error type 13 (specific pattern item): cause: 0x17ff555c0, Invalid VF ID for FDIR.: Invalid argument testpmd> # ../../usertools/dpdk-devbind.py -u 0000:08:06.0 0000:08:00.0 # modprobe uio # insmod ../kmod/igb_uio.ko # ../../usertools/dpdk-devbind.py -b igb_uio 0000:08:00.0 # ../../usertools/dpdk-devbind.py -b igb_uio 0000:08:06.0 testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 0 / end Caught error type 13 (specific pattern item): cause: 0x17ff535c0, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end Caught error type 13 (specific pattern item): cause: 0x17ff535c0, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end Caught error type 13 (specific pattern item): cause: 0x17ff535c0, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 0 / end Caught error type 13 (specific pattern item): cause: 0x17ff535c0, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 proto is 17 / vf id is 0 / end actions queue index 0 / end Caught error type 13 (specific pattern item): cause: 0x17ff53500, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 proto is 17 / vf id is 0 / end actions queue index 1 / end Caught error type 13 (specific pattern item): cause: 0x17ff53440, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end Caught error type 13 (specific pattern item): cause: 0x17ff53380, Invalid VF ID for FDIR.: Invalid argument testpmd> flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 0 / end Caught error type 13 (specific pattern item): cause: 0x17ff532c0, Invalid VF ID for FDIR.: Invalid argument testpmd> ```
Not a bug. Correct use steps： 1. bind the pf to dpdk driver:: ./usertools/dpdk-devbind.py -b igb_uio 1b:00.0 2. add one vf on dpdk pf, then bind the vf to vfio-pci:: echo 1 >/sys/bus/pci/devices/0000:1b:00.0/max_vfs ./usertools/dpdk-devbind.py -b igb_uio 1b:02.0 3. launch the app ``testpmd`` with the following arguments:: ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 1b:00.0 --file-prefix=pf – -i --rxq=4 --txq=4 testpmd> set verbose 1 testpmd> start ./x86_64-native-linuxapp-gcc/app/testpmd -c 0xf -n 4 -w 1b:02.0 --file-prefix=vf – -i --rxq=4 --txq=4 testpmd> set verbose 1 testpmd> start 4. create filter rule and sent pkg:: flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 255 / vf id is 0 / end actions queue index 1 / end sendp([Ether(dst='3C:FD:FE:DC:F1:88')/IP(src='11.11.11.11',dst='12.12.12.12',proto=255)/("X"*480)], iface="enp27s0f1", count=1) 5. flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 / udp / vf id is 0 / end actions queue index 1 / end sendp([Ether(dst='3C:FD:FE:DC:F1:88')/IP(src='11.11.11.11',dst='12.12.12.12',proto=17)/("X"*480)],iface="enp27s0f1", count=1) Note: ① dst mac is a random mac address, not a pf mac ② proto is 17 means UDP, so create the following rule and send the packet with proto is 17 cannot matched。 flow create 0 ingress transfer pattern eth / ipv4 src is 11.11.11.11 dst is 12.12.12.12 proto is 17 / vf id is 0 / end actions queue index 1 / end sendp([Ether(dst='3C:FD:FE:DC:F1:88')/IP(src='11.11.11.11',dst='12.12.12.12',proto=17)/("X"*480)], iface="enp27s0f1", count=1)
Marking this as fixed `confirmed` rather than `resolved-invalid` due to the following reasons 1. ticket is opened on DPDK 20.02 on 13 Apr 2020 2. The documentation available for FDIR on X710 was not correct 3. missing of `transfer` verb in the documentation of testpmd and tests. 4. Patches and fixes applied on top of X710 before testing on 30 Dec 2020. 5. Not mentioned in the comment whether it works for `DPDK 20.02` 6. Have not confirmed if the VF can be used with any other application other than testpmd. @Ajith will test and try on DPDK 20.02 but with much lower priority to confirm with the steps shared from `Steve Yang` and update the ticket.
Vipin, Do you have any update on the tests? Thanks
Test plan has been updated, please have try. https://doc.dpdk.org/dts/test_plans/generic_flow_api_test_plan.html