[dpdk-dev] Intel 82599ES send packets failed on dpdk-17.05.1 after traffic testing
zimeiw
zimeiw at 163.com
Wed Aug 23 13:47:14 CEST 2017
hi,
I have tested below dpdk version, after dpdk-17.05 version, ixgbe nic would send fail. the logs is same as below.
dpdk-17.02 ---- ok
dpdk-17.02.1 ---- ok
dpdk-17.05 ---- nok
dpdk-17.05.1 ---- nok
dpdk-17.08 ---- nok
PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 tx_tail=63 nb_tx=11
PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 tx_last=63
PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 nb_free= 0 (port=0 queue=0)
PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
tx burst failed, ret 0 , n 16, Success
PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63 tx_last=63
PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 nb_free= 0 (port=0 queue=0)
PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
tx burst failed, ret 0 , n 16, Success
PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=54 tx_first=63 tx_last=63
PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1 nb_free= 0 (port=0 queue=0)
PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
tx burst failed, ret 0 , n 5, Success
PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
--
Best Regards,
zimeiw
At 2017-08-23 17:04:44, "Ferruh Yigit" <ferruh.yigit at intel.com> wrote:
>On 8/23/2017 6:23 AM, zimeiw wrote:
>> hi,
>>
>> Any comments?
>>
>> is it possible NIC haredware issue? or NIC firmware version isn't match
>> dpdk driver?
>
>AFAIK for there isn't this kind of dependency for ixgbe. ixgbe driver
>maintainer cc'ed for more information.
>
>>
>> --
>> Best Regards,
>> zimeiw
>>
>> At 2017-08-23 08:52:38, "zimeiw" <zimeiw at 163.com> wrote:
>>
>> hi,
>>
>> "disable tso" logs were printed by my app.
>>
>> dpdk-17.05.1 is download from dpdk website, it is stable release.
>>
>>
>> --
>> Best Regards,
>> zimeiw
>>
>>
>> At 2017-08-22 22:23:54, "Ferruh Yigit" <ferruh.yigit at intel.com <mailto:ferruh.yigit at intel.com>> wrote:
>> >On 8/22/2017 2:51 PM, zimeiw wrote:
>> >> hi,
>> >> =====
>> >> check the dpdk default config, CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP isn't
>> >> enable.
>> >> #
>> >> # Turn off Tx preparation stage
>> >> #
>> >> # Warning: rte_ethdev_tx_prepare() can be safely disabled only if using a
>> >> # driver which do not implement any Tx preparation.
>> >> #
>> >> CONFIG_RTE_ETHDEV_TX_PREPARE_NOOP=n
>> >>
>> >> ====
>> >> I enable IXGBE debug for dpdk-17.05.1 and dpdk-17.02. the logs are in
>> >> the attachment.
>> >> app with dpdk-17.02 works ok.
>> >> app with dpdk-17.05.1 has below issue. it seems TX descriptor 95 can't
>> >> be handled by NIC?
>> >
>> >This is interesting, no idea what is going on, trying dpdk-17.05 can be
>> >an option, to be sure if this issue is in the original release or in the
>> >stable tree?
>> >
>> >btw, in 17.05.01, there are "disable tso" logs, do you know where they
>> >are coming from?
>> >
>> >>
>> >>
>> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 tx_tail=63 nb_tx=11
>> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
>> >
>> >Descriptor Done bit is not set, HW says descriptor is still in use, so
>> >cleaning descriptor fails.
>> >
>> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63
>> >> tx_last=63
>> >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1
>> >> nb_free= 0 (port=0 queue=0)
>> >
>> >And after a while there is no free descriptor left and Tx fails.
>> >
>> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
>> >> tx burst failed, ret 0 , n 16, nb_prep 0 , Success
>> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
>> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=205 tx_first=63
>> >> tx_last=63
>> >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1
>> >> nb_free= 0 (port=0 queue=0)
>> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
>> >> tx burst failed, ret 0 , n 16, nb_prep 0 , Success
>> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
>> >> PMD: ixgbe_xmit_pkts(): port_id=0 queue_id=0 pktlen=54 tx_first=63
>> >> tx_last=63
>> >> PMD: ixgbe_xmit_pkts(): Not enough free TX descriptors nb_used= 1
>> >> nb_free= 0 (port=0 queue=0)
>> >> PMD: ixgbe_xmit_cleanup(): TX descriptor 95 is not done(port=0 queue=0)
>> >> tx burst failed, ret 0 , n 5, nb_prep 0 , Success
>> >>
>> >>
>> >>
>> >> --
>> >> Best Regards,
>> >> zimeiw
>> >>
>> >>
>> >> At 2017-08-22 20:13:57, "Ferruh Yigit" <ferruh.yigit at intel.com <mailto:ferruh.yigit at intel.com>> wrote:
>> >>>On 8/22/2017 12:00 PM, zimeiw wrote:
>> >>>> hi,
>> >>>>
>> >>>> i set .txq_flags = 0.
>> >>>> and change the "send burst" code as below.
>> >>>> and printf some logs.
>> >>>> according to the logs, rte_eth_tx_burst() return 0, no error happen in
>> >>>> rte_eth_tx_burst().
>> >>>>
>> >>>> ====================
>> >>>> struct rte_mbuf **m_table;
>> >>>> int ret;
>> >>>> uint16_t queueid;
>> >>>> uint16_t nb_prep;
>> >>>>
>> >>>> queueid = qconf->tx_queue_id[port];
>> >>>> m_table = (struct rte_mbuf **)qconf->tx_mbufs[port].m_table;
>> >>>>
>> >>>> nb_prep = rte_eth_tx_prepare(port, queueid, m_table, n);
>> >>>> if (nb_prep != n)
>> >>>> printf("Preparing packet burst %d to transmit failed: %s\n",
>> >>>> nb_prep, rte_strerror(rte_errno));
>> >>>
>> >>>m->ol_flags should request offloading (PKT_TX_...), otherwise prepare
>> >>>does not do the work.
>> >>>
>> >>>but m->ol_flags is not changed, if this is working with 17.02, issue can
>> >>>be something else.
>> >>>
>> >>>You may try debugging ixgbe PMD to figure out at which point it gives an
>> >>>error.
>> >>>
>> >>>>
>> >>>> ret = rte_eth_tx_burst(port, queueid, m_table, nb_prep);
>> >>>> if (unlikely(ret < n))
>> >>>> {
>> >>>> printf("tx burst failed, ret %d , n %d, nb_prep %d , %s \n",
>> >>>> ret, n, nb_prep, rte_strerror(rte_errno) );
>> >>>> do
>> >>>> {
>> >>>> rte_pktmbuf_free(m_table[ret]);
>> >>>> } while (++ret < n);
>> >>>> }
>> >>>> =======================
>> >>>>
>> >>>> =====log======
>> >>>> Checking link status done
>> >>>> Port 0 Link Up - speed 10000 Mbps - full-duplex
>> >>>> USER8: main loop on lcore 1
>> >>>> USER8: -- lcoreid=1 portid=0 rxqueueid=0
>> >>>> nb ports 1 hz: 1698027100
>> >>>> tx burst failed, ret 2 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>> tx burst failed, ret 0 , n 16, nb_prep 16 , Success
>> >>>>
>> >>>> ==========NIC firware version =======
>> >>>> # ethtool -i enp2s0f1
>> >>>> driver: ixgbe
>> >>>> version: 4.2.1-k
>> >>>> firmware-version: 0x61bd0001
>> >>>> expansion-rom-version:
>> >>>> bus-info: 0000:02:00.1
>> >>>> supports-statistics: yes
>> >>>> supports-test: yes
>> >>>> supports-eeprom-access: yes
>> >>>> supports-register-dump: yes
>> >>>> supports-priv-flags: no
>> >>>>
>> >>>>
>> >>>> --
>> >>>> Best Regards,
>> >>>> zimeiw
>> >>>>
>> >>>>
>> >>>> At 2017-08-22 17:31:56, "Ferruh Yigit" <ferruh.yigit at intel.com <mailto:ferruh.yigit at intel.com>> wrote:
>> >>>>>On 8/22/2017 9:30 AM, zimeiw wrote:
>> >>>>>> hi,
>> >>>>>>
>> >>>>>> no use rte_eth_tx_prepare()?
>> >>>>>>
>> >>>>>> I set txq_flags value as below, may it cause this issue?
>> >>>>>>
>> >>>>>> .txq_flags = ~ETH_TXQ_FLAGS_NOXSUMS
>> >>>>>
>> >>>>>So you are enabling all csum offloading, for Tx path this requires
>> >>>>>packets to be prepared before sending to the NIC.
>> >>>>>
>> >>>>>Can you please test with rte_eth_tx_prepare()? testpmd has the sample usage.
>> >>>>>
>> >>>>>Thanks,
>> >>>>>ferruh
>> >>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Best Regards,
>> >>>>>> zimeiw
>> >>>>>>
>> >>>>>>
>> >>>>>> At 2017-08-22 15:49:56, "Ferruh Yigit" <ferruh.yigit at intel.com <mailto:ferruh.yigit at intel.com>> wrote:
>> >>>>>>>On 8/22/2017 5:45 AM, zimeiw wrote:
>> >>>>>>>> hi,
>> >>>>>>>> My test env is dell T430 server, NIC is intel 82599ES.
>> >>>>>>>> I integrate my app with dpdk-17.05.1 version. after do some tcp traffic testing, app can receive packets from the 82599 NIC, but 82599 NIC send packets failed.
>> >>>>>>>> But my app with dpdk-17.02 works well.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> If any change with ixgbe driver? or need some additional configuration for the NIC?
>> >>>>>>>
>> >>>>>>>tx_pkt_prepare added but it has been added on 17.02 which seems working
>> >>>>>>>for your case.
>> >>>>>>>
>> >>>>>>>Just to double check, are you using rte_eth_tx_prepare()?
>> >>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Thanks.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>>
>> >>>>>>>> Best Regards,
>> >>>>>>>> zimeiw
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
More information about the dev
mailing list