Bug 1406
Summary: | "net/ice" and "net/i40e" have not implemented outer UDP checksum offload, but the capability flag has been set. | ||
---|---|---|---|
Product: | DPDK | Reporter: | Jun Wang (junwang01) |
Component: | ethdev | Assignee: | Bruce Richardson (bruce.richardson) |
Status: | CONFIRMED --- | ||
Severity: | normal | CC: | david.marchand |
Priority: | Normal | ||
Version: | 23.11 | ||
Target Milestone: | --- | ||
Hardware: | All | ||
OS: | All |
Description
Jun Wang
2024-03-22 11:30:37 CET
Can you suggest a simple way to reproduce this issue and to validate any fix? Is it enough to try and send an mbuf with UDP checksum offloading and verify that, or is more setup for tunneling required? I think vxlan or geneve overlay is required for this. I'm using ovs-dpdk's vhost-user-client for virtual machines, and the communication between two physical nodes is done via a geneve tunnel.On the ovs side, I can see that RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM is set, but when I actually capture packets on the receiving end, the outer checksum is incorrect. Disabling RTE_ETH_TX_OFFLOAD_OUTER_UDP_CKSUM on the ovs side seems to resolve the issue,As shown in [1]. Additionally, after inspecting the DPDK driver code, it seems that it only advertises the feature but doesn't actually implement it,As shown in [2]. [1]https://patchwork.ozlabs.org/project/openvswitch/patch/1711095597-5729-1-git-send-email-junwang01@cestc.cn/ [2]https://github.com/DPDK/dpdk/blob/main/drivers/net/i40e/i40e_rxtx.c#L301 According to Intel(R) Ethernet Controller X710/ XXV710/XL710 Datasheet, section 8.4.4.2, "Tunneling UDP headers and GRE header are not offloaded while the X710/XXV710/XL710 leaves their checksum field as is".So for the X710 series, DPDK should not advertise support for outer UDP offload, which appears to be a bug in DPDK. https://cdrdv2-public.intel.com/332464/332464_710_Series_Datasheet_v_4_1.pdf According to Intel(R) Ethernet Controller E810 Datasheet,section 10.5.8.3, The tunneling UDP checksum offload appears to be supported. https://www.intel.com/content/www/us/en/content-details/613875/intel-ethernet-controller-e810-datasheet.html Indeed, I had read the datasheets and came to the same conclusions. I spent some time on this topic and I think we are missing the setup of the pseudo header checksum on the dpdk side: diff --git a/lib/net/rte_net.h b/lib/net/rte_net.h index ef3ff4c6fd..73a6618a66 100644 --- a/lib/net/rte_net.h +++ b/lib/net/rte_net.h @@ -121,7 +121,8 @@ rte_net_intel_cksum_flags_prepare(struct rte_mbuf *m, uint64_t ol_flags) * no offloads are requested. */ if (!(ol_flags & (RTE_MBUF_F_TX_IP_CKSUM | RTE_MBUF_F_TX_L4_MASK | RTE_MBUF_F_TX_TCP_SEG | - RTE_MBUF_F_TX_UDP_SEG | RTE_MBUF_F_TX_OUTER_IP_CKSUM))) + RTE_MBUF_F_TX_UDP_SEG | RTE_MBUF_F_TX_OUTER_IP_CKSUM | + RTE_MBUF_F_TX_OUTER_UDP_CKSUM))) return 0; if (ol_flags & (RTE_MBUF_F_TX_OUTER_IPV4 | RTE_MBUF_F_TX_OUTER_IPV6)) { @@ -135,6 +136,22 @@ rte_net_intel_cksum_flags_prepare(struct rte_mbuf *m, uint64_t ol_flags) struct rte_ipv4_hdr *, m->outer_l2_len); ipv4_hdr->hdr_checksum = 0; } + if (ol_flags & RTE_MBUF_F_TX_OUTER_UDP_CKSUM) { + if (ol_flags & RTE_MBUF_F_TX_OUTER_IPV4) { + ipv4_hdr = rte_pktmbuf_mtod_offset(m, struct rte_ipv4_hdr *, + m->outer_l2_len); + udp_hdr = (struct rte_udp_hdr *)((char *)ipv4_hdr + + m->outer_l3_len); + udp_hdr->dgram_cksum = rte_ipv4_phdr_cksum(ipv4_hdr, m->ol_flags); + } else { + ipv6_hdr = rte_pktmbuf_mtod_offset(m, struct rte_ipv6_hdr *, + m->outer_l2_len); + /* non-TSO udp */ + udp_hdr = rte_pktmbuf_mtod_offset(m, struct rte_udp_hdr *, + m->outer_l2_len + m->outer_l3_len); + udp_hdr->dgram_cksum = rte_ipv6_phdr_cksum(ipv6_hdr, m->ol_flags); + } + } } /* With a E810 + OVS (vxlan + tso): Before: 10:00:27.152204 50:7c:6f:3c:0c:26 > 50:7c:6f:3c:10:5a, ethertype 802.1Q (0x8100), length 148: vlan 200, p 0, ethertype IPv6, (hlim 64, next-header UDP (17) payload length: 90) 2001::2.36209 > 2001::1.vxlan: [bad udp cksum 0x35cc -> 0xf55b!] VXLAN, flags [I] (0x08), vni 1 52:54:00:00:11:01 > 36:44:f8:6f:0b:1a, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 172.31.22.2.targus-getdata1 > 172.31.22.1.41186: Flags [S.], cksum 0x0ee5 (correct), seq 3569139204, ack 2007533520, win 26960, options [mss 1360,sackOK,TS val 998328331 ecr 3390027474,nop,wscale 7], length 0 After: 10:09:51.088760 50:7c:6f:3c:0c:26 > 50:7c:6f:3c:10:5a, ethertype 802.1Q (0x8100), length 148: vlan 200, p 0, ethertype IPv6, (hlim 64, next-header UDP (17) payload length: 90) 2001::2.38344 > 2001::1.vxlan: [udp sum ok] VXLAN, flags [I] (0x08), vni 1 52:54:00:00:11:01 > 36:44:f8:6f:0b:1a, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) 172.31.22.2.targus-getdata1 > 172.31.22.1.53988: Flags [S.], cksum 0xb25c (correct), seq 1558118886, ack 381507235, win 26960, options [mss 1360,sackOK,TS val 998892263 ecr 3390639231,nop,wscale 7], length 0 I posted a series trying to fix issues related to outer udp checksum. https://patchwork.dpdk.org/project/dpdk/list/?series=31687&state=%2A&archive=both I've used your V2 version submitted to the DPDK community, and everything seems to be working fine now. It's great! https://patches.dpdk.org/project/dpdk/list/?series=31689 |