[dpdk-dev] [PATCH] virtio: tx with can_push when VERSION_1 is set

[Pierre Pfister (ppfister)]

Hello Maxime,

Sorry for the late reply.


> Le 8 nov. 2016 à 10:44, Maxime Coquelin <maxime.coquelin at redhat.com> a écrit :
> 
> Hi Pierre,
> 
> On 11/08/2016 10:31 AM, Pierre Pfister (ppfister) wrote:
>> Current virtio driver advertises VERSION_1 support,
>> but does not handle device's VERSION_1 support when
>> sending packets (it looks for ANY_LAYOUT feature,
>> which is absent).
>> 
>> This patch enables 'can_push' in tx path when VERSION_1
>> is advertised by the device.
>> 
>> This significantly improves small packets forwarding rate
>> towards devices advertising VERSION_1 feature.
> I think it depends whether offloading is enabled or not.
> If no offloading enabled, I measured significant drop.
> Indeed, when no offloading is enabled, the Tx path in Virtio
> does not access the virtio header before your patch, as the header is memset to zero at device init time.
> With your patch, it gets memset to zero at every transmit in the hot
> path.

Right. On the virtio side that is true, but on the device side, we have to access the header anyway.
And accessing two descriptors (with the address resolution and memory fetch which comes with it)
is a costy operation compared to a single one.
In the case indirect descriptors are used, this is 1 desc access instead or 3.
And in the case chained descriptors are used, this doubles the number of packets that you can put in your queue.

Those are the results in my PHY -> VM (testpmd) -> PHY setup
Traffic is flowing bidirectionally. Numbers are for lossless-rates.

When chained buffers are used for dpdk's TX: 2x2.13Mpps
When indirect descriptors are used for dpdk's TX: 2x2.38Mpps
When shallow buffers are used for dpdk's TX (with this patch): 2x2.42Mpps

I must also note that qemu 2.5 does not seem to deal with VERSION_1 and ANY_LAYOUT correctly.
The patch I am proposing here works for qemu 2.7, but with qemu 2.5, testpmd still behaves as if ANY_LAYOUT (or VERSION_1) was not available. This is not catastrophic. But just note that you will not see performance in some cases with qemu 2.5.

Cheers

- Pierre

> 
> With offloading enabled, it does makes sense though, as the header will
> be accessed.
> 
> This patch is for v17.02 anyway, and we may provide a way to enable and
> disable features at Virtio PMD init time by this release.
> 
> Thanks,
> Maxime
> 
>> 
>> Signed-off-by: Pierre Pfister <ppfister at cisco.com>
>> ---
>> drivers/net/virtio/virtio_rxtx.c | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
>> index 724517e..2fe0338 100644
>> --- a/drivers/net/virtio/virtio_rxtx.c
>> +++ b/drivers/net/virtio/virtio_rxtx.c
>> @@ -925,7 +925,8 @@ virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, uint16_t nb_pkts)
>>                }
>> 
>>                /* optimize ring usage */
>> -               if (vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) &&
>> +               if ((vtpci_with_feature(hw, VIRTIO_F_ANY_LAYOUT) ||
>> +                     vtpci_with_feature(hw, VIRTIO_F_VERSION_1)) &&
>>                    rte_mbuf_refcnt_read(txm) == 1 &&
>>                    RTE_MBUF_DIRECT(txm) &&
>>                    txm->nb_segs == 1 &&
>> --
>> 2.7.4 (Apple Git-66)
>>