[dpdk-dev] net/virtio: fix use_msix get the wrong value

  When running l3fwd-power to test virtio rxq interrupt using vfio
pci noiommu mode, startup fails. In the function virtio_read_caps,
the code if (flags & PCI_MSIX_ENABLE) intends to double check
if vfio msix is enabled or not. However, it is indeed not valid.
Come back to l3fwd-power, use_msix is not assigned to the right
value "1". The patch fixes the issue.

Fixes: cb482cb3a305 ("net/virtio: fix MAC address read")
Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com>
---
 drivers/net/virtio/virtio_pci.c | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)
  

On Tue, Oct 31, 2017 at 05:44:56PM +0800, Zhiyong Yang wrote:
> When running l3fwd-power to test virtio rxq interrupt using vfio
> pci noiommu mode, startup fails.

What errors have you met?

> In the function virtio_read_caps,
> the code if (flags & PCI_MSIX_ENABLE) intends to double check
> if vfio msix is enabled or not. However, it is indeed not valid.
> Come back to l3fwd-power, use_msix is not assigned to the right
> value "1". The patch fixes the issue.

Jianfeng, could you review it? You are the last author changed the code.
Will it break your case?

	--yliu

> Fixes: cb482cb3a305 ("net/virtio: fix MAC address read")
> Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com>
> ---
>  drivers/net/virtio/virtio_pci.c | 11 +----------
>  1 file changed, 1 insertion(+), 10 deletions(-)
> 
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 55b717c03..be5b07a58 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -580,8 +580,6 @@ get_cfg_addr(struct rte_pci_device *dev, struct virtio_pci_cap *cap)
>  	return base + offset;
>  }
>  
> -#define PCI_MSIX_ENABLE 0x8000
> -
>  static int
>  virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
>  {
> @@ -609,14 +607,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
>  		}
>  
>  		if (cap.cap_vndr == PCI_CAP_ID_MSIX) {
> -			/* Transitional devices would also have this capability,
> -			 * that's why we also check if msix is enabled.
> -			 * 1st byte is cap ID; 2nd byte is the position of next
> -			 * cap; next two bytes are the flags.
> -			 */
> -			uint16_t flags = ((uint16_t *)&cap)[1];
> -
> -			if (flags & PCI_MSIX_ENABLE)
> +			if (dev->intr_handle.type == RTE_INTR_HANDLE_VFIO_MSIX)
>  				hw->use_msix = 1;
>  		}
>  
> -- 
> 2.13.3
  

Hi Yuanhan,

> -----Original Message-----
> From: Yuanhan Liu [mailto:yliu@fridaylinux.org]
> Sent: Wednesday, November 1, 2017 11:40 PM
> To: Yang, Zhiyong <zhiyong.yang@intel.com>
> Cc: dev@dpdk.org; maxime.coquelin@redhat.com; Tan, Jianfeng
> <jianfeng.tan@intel.com>
> Subject: Re: [PATCH] net/virtio: fix use_msix get the wrong value
> 
> On Tue, Oct 31, 2017 at 05:44:56PM +0800, Zhiyong Yang wrote:
> > When running l3fwd-power to test virtio rxq interrupt using vfio pci
> > noiommu mode, startup fails.
> 
> What errors have you met?
> 

EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
EAL:   using IOMMU type 8 (No-IOMMU)
EAL: Ignore mapping IO port bar(0)
Promiscuous mode selected
numa is disabled 
soft parse-ptype is enabled
Initializing port 0 ... EAL: Error - exiting with code: 1
  Cause: Cannot configure not existed rxq: port=0

Thanks
Zhiyong
  

On 10/31/2017 5:44 PM, Zhiyong Yang wrote:
> When running l3fwd-power to test virtio rxq interrupt using vfio
> pci noiommu mode, startup fails. In the function virtio_read_caps,
> the code if (flags & PCI_MSIX_ENABLE) intends to double check
> if vfio msix is enabled or not. However, it is indeed not valid.
> Come back to l3fwd-power, use_msix is not assigned to the right
> value "1". The patch fixes the issue.
>
> Fixes: cb482cb3a305 ("net/virtio: fix MAC address read")
> Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com>
> ---
>   drivers/net/virtio/virtio_pci.c | 11 +----------
>   1 file changed, 1 insertion(+), 10 deletions(-)
>
> diff --git a/drivers/net/virtio/virtio_pci.c b/drivers/net/virtio/virtio_pci.c
> index 55b717c03..be5b07a58 100644
> --- a/drivers/net/virtio/virtio_pci.c
> +++ b/drivers/net/virtio/virtio_pci.c
> @@ -580,8 +580,6 @@ get_cfg_addr(struct rte_pci_device *dev, struct virtio_pci_cap *cap)
>   	return base + offset;
>   }
>   
> -#define PCI_MSIX_ENABLE 0x8000
> -
>   static int
>   virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
>   {
> @@ -609,14 +607,7 @@ virtio_read_caps(struct rte_pci_device *dev, struct virtio_hw *hw)
>   		}
>   
>   		if (cap.cap_vndr == PCI_CAP_ID_MSIX) {
> -			/* Transitional devices would also have this capability,
> -			 * that's why we also check if msix is enabled.
> -			 * 1st byte is cap ID; 2nd byte is the position of next
> -			 * cap; next two bytes are the flags.
> -			 */
> -			uint16_t flags = ((uint16_t *)&cap)[1];
> -
> -			if (flags & PCI_MSIX_ENABLE)
> +			if (dev->intr_handle.type == RTE_INTR_HANDLE_VFIO_MSIX)
>   				hw->use_msix = 1;
>   		}
>   

The real issue is that, if vfio (noiommu) is used, msix enable 
(rte_intr_enable) is after the msix detection. IMO, we shall try to move 
msix enable ahead for virtio.
- igb_uio do that in the open().
- uio_pci_generic does not.
- vfio-pci does not.

For previous fix on mmio position adjustment, we might need to change to 
ask if msix is enabled each time.

Thanks,
Jianfeng