[dpdk-dev] NIC Stops Transmitting

[jinho hwang]

On Fri, Jul 26, 2013 at 4:04 PM, Scott Talbert <swt at techie.net> wrote:
> On Fri, 26 Jul 2013, Stephen Hemminger wrote:
>
>>> I'm writing an application using DPDK that transmits a large number of
>>> packets (it doesn't receive any).  When I transmit at 2 Gb/sec,
>>> everything
>>> will run fine for several seconds (receiver is receiving at correct
>>> rate),
>>> but then the NIC appears to get 'stuck' and doesn't transmit any more
>>> packets.  In this state, rte_eth_tx_burst() is returning zero (suggesting
>>> that there are no available transmit descriptors), but even if I sleep()
>>> for a second and try again, rte_eth_tx_burst() still returns 0.  It
>>> almost
>>> appears as if a packet gets stuck in the transmit ring and keeps
>>> everything from flowing.  I'm using an Intel 82599EB NIC.
>>>
>> Make sure there is enough memory for mbufs.
>> Also what is your ring size and transmit free threshold?
>> It is easy to instrument the driver to see where it is saying "no space
>> left"
>> Also be careful with threshold values, many values of
>> pthresh/hthresh/wthresh
>> don't work. I would check the Intel reference manual for your hardware.
>
>
> Thanks for the tips.  I don't think I'm running out of mbufs, but I'll check
> that again.  I am using these values from one of the examples - which claim
> to be correct for the 82599EB.
>
> /*
>  * These default values are optimized for use with the Intel(R) 82599 10 GbE
>  * Controller and the DPDK ixgbe PMD. Consider using other values for other
>  * network controllers and/or network drivers.
>  */
> #define TX_PTHRESH 36 /**< Default values of TX prefetch threshold reg. */
> #define TX_HTHRESH 0  /**< Default values of TX host threshold reg. */
> #define TX_WTHRESH 0  /**< Default values of TX write-back threshold reg. */
>
> static const struct rte_eth_txconf tx_conf = {
>     .tx_thresh = {
>         .pthresh = TX_PTHRESH,
>         .hthresh = TX_HTHRESH,
>         .wthresh = TX_WTHRESH,
>     },
>     .tx_free_thresh = 0, /* Use PMD default values */
>     .tx_rs_thresh = 0, /* Use PMD default values */
> };
>
> /*
>  * Configurable number of RX/TX ring descriptors
>  */
> #define RTE_TEST_TX_DESC_DEFAULT 512
> static uint16_t nb_txd = RTE_TEST_TX_DESC_DEFAULT;
>

Scott,

I am wondering whether you use multiple cores accessing the same
receive queue. I had this problem before, but after I make the same
number of receiving queues as the number of receiving cores, the
problem disappeared. I did not dig more since I did not care how many
receive queues I have did not matter.

Jinho