[dpdk-dev] Passing VLAN traffic via l2fwd

[James Yu]

Some more discovery.

That flag PKT_TX_VLAN_PKT eventually send the following command to the
chips (ixgbe_xmit_pkts(), tx_desc_vlan_flags_to_cmdtype()) . Is this
actually adding a VLAN tag during tx ?



#define IXGBE_ADVTXD_DCMD_VLE           IXGBE_TXD_CMD_VLE  /* VLAN pkt
enable */

#define IXGBE_TXD_CMD_VLE       0x40000000 /* Add VLAN tag */

After I intentionally remove the tag before sending out the traffic, the
Spirent can receive tagged packets. I am not sure the DPDK codes is adding
a tag.  I think the ixgbevf on the host is adding VLAN tag based on the
configured tag ID.

Is this expected ? How can I configure the host to not add tag since
traffic from the VM is already tagged. Otherwise it becomes double tagged.

Thanks

James


On Fri, Jan 24, 2014 at 6:07 AM, James Yu <ypyu2011 at gmail.com> wrote:

> Stephen,
>
> I added that to the l2fwd main.c and it is still not received by the
> Spirent. I obtained the stat using rte_eth_stats_get() and printout the
> following when I send only one VLAN pkt
> ipackets: increased by 1
> opackets: no change (pkt not sent out by the device)
> ierrors, oerrors, otxerror, otxdiscard: all 0
>
> The same thing with 10 packets which increase 10 ipackets counts.
>
> I found out that the rte_eth_stats_get() obtains the stats from
> ixgbevf_dev_stats_get(). Is this expected for SR-IOV that uses
> ixgbe_ethdev.c ?
> If that's correct, I think this tells me that the tagged packet is
> received by the VF PMD codes but not sent out and has no error or packets
> dropped.
>
> So it was not sent out of the VF PMD. But why there is not error or packet
> dropped. Where did it go ? I am trying to find out where it got dropped and
> how to fix it.
>
> I tried DPDK1.5.2r1. It exhibits the same issue even with the patch.
>
> Thanks
>
> James
>
>
>
> On Thu, Jan 23, 2014 at 1:43 PM, Stephen Hemminger <
> stephen at networkplumber.org> wrote:
>
>> On Thu, 23 Jan 2014 11:27:14 -0800
>> James Yu <ypyu2011 at gmail.com> wrote:
>>
>> > Hi Daniel,
>> >
>> > Untagged traffic could be looped back. I am doing a one-directional loop
>> > back. In the Frame tabe of the StreamBlock editor, I did the following
>> > stream1:
>> >      destination address: MAC address of the receiving port of the guest
>> > VM. (obtained from the debug print out when l2fwd is started, or
>> ifconfig
>> > before l2fwd is started)
>> > stream2:
>> >      destination address: MAC address of the sending port of the guest
>> VM
>> > (obtained from the debug print out when l2fwd is started, or ifconfig
>> > before l2fwd is started)
>> >
>> >
>> > The only thing I add is to tag both the SR-IOV VF ports on the host with
>> > the same tag using ip command and add that tag number as the VID to the
>> > Spirent streams. After that that traffic can reach l2fwd which reported
>> > forwarded the packets on the periodic printout. However the packets did
>> not
>> > reach the Spirent.
>> >
>> > Also it is exercising rte_ixgbevf_pmd codes as indicated during starting
>> > l2fwd.
>> >
>> > EAL: Master core 0 is ready (tid=b77ab7d0)
>> >
>> > EAL: Core 1 is ready (tid=adffeb70)
>> >
>> > EAL: probe driver: 8086:100e rte_em_pmd
>> >
>> > EAL: probe driver: 8086:100e rte_em_pmd
>> >
>> > *EAL: probe driver: 8086:10ed rte_ixgbevf_pmd*
>> > I wonder do I have to change any codes in the DPDK library or call
>> certain
>> > routines to setup the VLAN in the DPDK stack during starting the l2fwd ?
>> >
>> >
>> > Were you able to send VLAN traffic from your iXia and looped back to
>> your
>> > iXia ?
>> >
>> >
>> > Thanks
>> >
>> > James
>> >
>> >
>> >
>> >
>> > On Thu, Jan 23, 2014 at 12:45 AM, Daniel Kaminsky <
>> > daniel.kaminsky at infinitelocality.com> wrote:
>> >
>> > > Hi James,
>> > >
>> > > Did you try a non-tagged traffic? I suspect this is the limitation of
>> the
>> > > l2fwd or the requirements of Spirent. From my experience with Ixia,
>> the L2
>> > > forward device must implement the ARP protocol (at least a subset of
>> it).
>> > >
>> > > Also, notice that the l2fwd example sets the destination MAC address
>> to
>> > > "02:00:00:00:00:xx" which the Spirent might doesn't like.
>> > >
>> > > Daniel
>> > >
>> > >
>> > > On Thu, Jan 23, 2014 at 7:08 AM, James Yu <ypyu2011 at gmail.com> wrote:
>> > >
>> > >> I could not pass tagged traffic through the l2fwd program running
>> inside a
>> > >> CentOS VM. The l2fwd program reported sending out all the packets
>> received
>> > >> from one port to the other port. But the outside Spirent tester
>> could not
>> > >> receive a packet at all. I am wondering maybe the tagged packets are
>> > >> dropped somewhere in the rte_em_pmd or in the DPDK library. Anyone
>> knows
>> > >> how to pass/send VLAN traffic through the DPDK ?
>> > >>
>> > >> With tagged SR-IOV ports, Spirent and guest VM CentOS could ping each
>> > >> other
>> > >> without using DPDK. Once I used DPDK 1.3.1r2, I encountered issues
>> with
>> > >> sending out tagged packets out of DPDK to the host.
>> > >>
>> > >> I have done the following:
>> > >>
>> > >> On RHEL6.1 host
>> > >> 1. setup the VLAN using "ip link set eth2 vf 0 vlan 3" for port 1
>> and "ip
>> > >> link set eth3 vf 0 vlan 3" for port 2
>> > >> [root at cent64x64-137189 ~]# ip link show eth2
>> > >> 96: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
>> UP
>> > >> qlen
>> > >> 1000
>> > >>     link/ether 90:e2:ba:2e:af:f0 brd ff:ff:ff:ff:ff:ff
>> > >>     vf 0 MAC 52:54:00:f9:9a:90, vlan 3
>> > >> [root at cent64x64-137189 ~]# ip link show eth3
>> > >> 98: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state
>> UP
>> > >> qlen
>> > >> 1000
>> > >>     link/ether 90:e2:ba:2e:af:f1 brd ff:ff:ff:ff:ff:ff
>> > >>     vf 0 MAC 52:54:00:30:37:98, vlan 3
>> > >>
>> > >> 2. virsh configure SR-IOV hostdev
>> > >>     <interface type='hostdev' managed='yes'>
>> > >>       <mac address='52:54:00:f9:9a:90'/>
>> > >>       <source>
>> > >>         <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
>> > >> function='0x0'/>
>> > >>       </source>
>> > >>       <address type='pci' domain='0x0000' bus='0x00' slot='0x08'
>> > >> function='0x0'/>
>> > >>     </interface>
>> > >>     <interface type='hostdev' managed='yes'>
>> > >>       <mac address='52:54:00:30:37:98'/>
>> > >>       <source>
>> > >>         <address type='pci' domain='0x0000' bus='0x1a' slot='0x10'
>> > >> function='0x1'/>
>> > >>       </source>
>> > >>       <address type='pci' domain='0x0000' bus='0x00' slot='0x09'
>> > >> function='0x0'/>
>> > >>     </interface>
>> > >>
>> > >> [root at cent64x64-137189 ~]# lspci |grep Eth
>> > >> 1a:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+
>> > >> Network Connection (rev 01)
>> > >> 1a:00.1 Ethernet controller: Intel Corporation 82599EB 10-Gigabit
>> SFI/SFP+
>> > >> Network Connection (rev 01)
>> > >> 1a:10.0 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >> 1a:10.1 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >>
>> > >>
>> > >> On guest VM
>> > >> 1. check all the ports are started up and check their corresponding
>> port
>> > >> type. Use the two SR-IOV ports with device type as ixgbevf  for l2fwd
>> > >> [root at VirtualADX ~]# lspci |grep Eth
>> > >> 00:03.0 Ethernet controller: Intel Corporation 82540EM Gigabit
>> Ethernet
>> > >> Controller (rev 03)
>> > >> 00:07.0 Ethernet controller: Intel Corporation 82540EM Gigabit
>> Ethernet
>> > >> Controller (rev 03)
>> > >> 00:08.0 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >> 00:09.0 Ethernet controller: Intel Corporation 82599 Ethernet
>> Controller
>> > >> Virtual Function (rev 01)
>> > >> 00:0a.0 Ethernet controller: Intel Corporation 82540EM Gigabit
>> Ethernet
>> > >> Controller (rev 03)
>> > >>
>> > >> 2. run this command:
>> > >>            /root/l2fwd -c 3 -n 1 -b 000:00:03.0  -b 000:00:07.0 -b
>> > >> 000:00:0a.0 -- -q 1 -p 3
>> > >>
>> > >>     Although the l2fwd forwards all received tagged packets, the
>> Spirent
>> > >> tester did not receive any packet at all.
>> > >>
>> > >
>> > >
>>
>> The VLAN offload bit in the mbuf is different for TX and RX.
>> Probably would work with something like this:
>>
>> diff --git a/examples/l2fwd/main.c b/examples/l2fwd/main.c
>> index 05ef330..70dbf85 100644
>> --- a/examples/l2fwd/main.c
>> +++ b/examples/l2fwd/main.c
>> @@ -289,6 +289,9 @@ l2fwd_simple_forward(struct rte_mbuf *m, unsigned
>> portid)
>>         /* src addr */
>>         ether_addr_copy(&l2fwd_ports_eth_addr[dst_port], &eth->s_addr);
>>
>> +       if (m->ol_flags & PKT_RX_VLAN_PKT)
>> +               m->ol_flags |= PKT_TX_VLAN_PKT;
>> +
>>         l2fwd_send_packet(m, (uint8_t) dst_port);
>>  }
>>
>>
>>
>>
>