[dpdk-stable] [dpdk-dev] [PATCH] nfp: handle packets with length 0 as usual ones

[Ferruh Yigit]

On 8/18/2017 5:23 PM, Alejandro Lucero wrote:
> 
> 
> On Fri, Aug 18, 2017 at 4:10 PM, Ferruh Yigit <ferruh.yigit at intel.com
> <mailto:ferruh.yigit at intel.com>> wrote:
> 
>     On 8/11/2017 11:05 AM, Alejandro Lucero wrote:
>     > A DPDK app could, whatever the reason, send packets with size 0.
>     > The PMD is not sending those packets, which does make sense,
>     > but the problem is the mbuf is not released either. That leads
>     > to mbufs not being available, because the app trusts the
>     > PMD will do it.
>     >
>     > Although this is a problem related to app wrong behaviour, we
>     > should harden the PMD in this regard. Not sending a packet with
>     > size 0 could be problematic, needing special handling inside the
>     > PMD xmit function. It could be a burst of those packets, which can
>     > be easily handled, but it could also be a single packet in a burst,
>     > what is harder to handle.
>     >
>     > It would be simpler to just send that kind of packets, which will
>     > likely be dropped by the hw at some point. The main problem is how
>     > the fw/hw handles the DMA, because a dma read to a hypothetical 0x0
>     > address could trigger an IOMMU error. It turns out, it is safe to
>     > send a descriptor with packet size 0 to the hardware: the DMA never
>     > happens, from the PCIe point of view.
>     >
>     > Signed-off-by: Alejandro Lucero <alejandro.lucero at netronome.com
>     <mailto:alejandro.lucero at netronome.com>>
>     > ---
>     >  drivers/net/nfp/nfp_net.c | 17 ++++++++++++-----
>     >  1 file changed, 12 insertions(+), 5 deletions(-)
>     >
>     > diff --git a/drivers/net/nfp/nfp_net.c b/drivers/net/nfp/nfp_net.c
>     > index 92b03c4..679a91b 100644
>     > --- a/drivers/net/nfp/nfp_net.c
>     > +++ b/drivers/net/nfp/nfp_net.c
>     > @@ -2094,7 +2094,7 @@ uint32_t nfp_net_txq_full(struct nfp_net_txq
>     *txq)
>     >                */
>     >               pkt_size = pkt->pkt_len;
>     >
>     > -             while (pkt_size) {
>     > +             while (pkt) {
>     >                       /* Copying TSO, VLAN and cksum info */
>     >                       *txds = txd;
>     >
>     > @@ -2126,17 +2126,24 @@ uint32_t nfp_net_txq_full(struct
>     nfp_net_txq *txq)
>     >                               txq->wr_p = 0;
>     >
>     >                       pkt_size -= dma_size;
>     > -                     if (!pkt_size) {
>     > +                     if (!pkt_size)
>     >                               /* End of packet */
>     >                               txds->offset_eop |= PCIE_DESC_TX_EOP;
>     > -                     } else {
>     > +                     else
>     >                               txds->offset_eop &=
>     PCIE_DESC_TX_OFFSET_MASK;
>     > -                             pkt = pkt->next;
>     > -                     }
>     > +
>     > +                     pkt = pkt->next;
>     >                       /* Referencing next free TX descriptor */
>     >                       txds = &txq->txds[txq->wr_p];
>     >                       lmbuf = &txq->txbufs[txq->wr_p].mbuf;
>     >                       issued_descs++;
>     > +
>     > +                     /* Double-checking if we have to use chained
>     mbuf.
>     > +                      * It seems there are some apps which could
>     wrongly
>     > +                      * have zeroed mbufs chained leading to send
>     null
>     > +                      * descriptors to the hw. */
>     > +                     if (!pkt_size)
>     > +                             break;
> 
>     For the case chained mbufs with all are zero size [1], won't this cause
>     next mbufs not freed because rte_pktmbuf_free_seg(*lmbuf) used?
> 
> 
> Good point. Being honest, we had the problem with mbufs and size 0, and
> this last check
> was not initially there. But we saw performance being low after the
> change, and the only thing
> which could explain it was this sort of chained mbufs. There was not
> mbuf allocation problem at
> all. It was like more (null) packets being sent to the hardware now.
> This last check solved the
> performance problem. 

I assume performance problem is with the chained mbufs with 0 size, I
believe this should be fixed in application, not in PMD level.

And if application is sending chained mbufs with 0 size, with above code
it will eventually be out off mbufs, since they are not freed, and same
problem will occur that this patch is trying to avoid, but perhaps in
longer run.

> 
> Once I have said that, I have to admit my explanation implies some
> serious problem when
> handling mbufs, and something the app is doing really badly, so I could
> understand someone
> saying this is hidden a serious problem and should not be there. 
> 
>     [1]
>     As you mentioned in the commit log, this not correct thing to do, but
>     since patch is trying to harden PMD for this wrong application
>     behavior..
> 
> 
> If you consider this last check should not be there, I'll be glad to
> remove it.
>  
> 
> 
>     >               }
>     >               i++;
>     >       }
>     >
> 
>