[dpdk-dev] [PATCH v8 0/5] add ifcvf vdpa driver

[Ferruh Yigit]

On 4/16/2018 4:34 PM, Xiao Wang wrote:
> IFCVF driver
> ============
> The IFCVF vDPA (vhost data path acceleration) driver provides support for the
> Intel FPGA 100G VF (IFCVF). IFCVF's datapath is virtio ring compatible, it
> works as a HW vhost backend which can send/receive packets to/from virtio
> directly by DMA. Besides, it supports dirty page logging and device state
> report/restore. This driver enables its vDPA functionality with live migration
> feature.
> 
> vDPA mode
> =========
> IFCVF's vendor ID and device ID are same as that of virtio net pci device,
> with its specific subsystem vendor ID and device ID. To let the device be
> probed by IFCVF driver, adding "vdpa=1" parameter helps to specify that this
> device is to be used in vDPA mode, rather than polling mode, virtio pmd will
> skip when it detects this message.
> 
> Container per device
> ====================
> vDPA needs to create different containers for different devices, thus this
> patch set adds some APIs in eal/vfio to support multiple container, e.g.
> - rte_vfio_create_container
> - rte_vfio_destroy_container
> - rte_vfio_bind_group
> - rte_vfio_unbind_group
> 
> By this extension, a device can be put into a new specific container, rather
> than the previous default container.
> 
> IFCVF vDPA details
> ==================
> Key vDPA driver ops implemented:
> - ifcvf_dev_config:
>   Enable VF data path with virtio information provided by vhost lib, including
>   IOMMU programming to enable VF DMA to VM's memory, VFIO interrupt setup to
>   route HW interrupt to virtio driver, create notify relay thread to translate
>   virtio driver's kick to a MMIO write onto HW, HW queues configuration.
> 
>   This function gets called to set up HW data path backend when virtio driver
>   in VM gets ready.
> 
> - ifcvf_dev_close:
>   Revoke all the setup in ifcvf_dev_config.
> 
>   This function gets called when virtio driver stops device in VM.
> 
> Change log
> ==========
> v8:
> - Rebase on HEAD.
> - Move vfio_group definition back to eal_vfio.h.
> - Return NULL when vfio group num/fd is not found, let caller handle that.
> - Fix wrong API name in commit log.
> - Rename bind/unbind function to rte_vfio_container_group_bind/unbind for
>   consistensy.
> - Add note for rte_vfio_container_create and rte_vfio_dma_map and fix typo
>   in comment.
> - Extract out the shared code snip of rte_vfio_dma_map and
>   rte_vfio_container_dma_map to avoid code duplication. So do for the unmap.
> 
> v7:
> - Rebase on HEAD.
> - Split the vfio patch into 2 parts, one for data structure extension, one for
>   adding new API.
> - Use static vfio_config array instead of dynamic alloating.
> - Change rte_vfio_container_dma_map/unmap's parameters to use (va, iova, len).
> 
> v6:
> - Rebase on master branch.
> - Document "vdpa" devarg in virtio documentation.
> - Rename ifcvf config option to CONFIG_RTE_LIBRTE_IFCVF_VDPA_PMD for
>   consistensy, and add it into driver documentation.
> - Add comments for ifcvf device ID.
> - Minor code cleaning.
> 
> v5:
> - Fix compilation in BSD, remove the rte_vfio.h including in BSD.
> 
> v4:
> - Rebase on Zhihong's latest vDPA lib patch, with vDPA ops names change.
> - Remove API "rte_vfio_get_group_fd", "rte_vfio_bind_group" will return the fd.
> - Align the vfio_cfg search internal APIs naming.
> 
> v3:
> - Add doc and release note for the new driver.
> - Remove the vdev concept, make the driver as a PCI driver, it will get probed
>   by PCI bus driver.
> - Rebase on the v4 vDPA lib patch, register a vDPA device instead of a engine.
> - Remove the PCI API exposure accordingly.
> - Move the MAX_VFIO_CONTAINERS definition to config file.
> - Let virtio pmd skips when a virtio device needs to work in vDPA mode.
> 
> v2:
> - Rename function pci_get_kernel_driver_by_path to rte_pci_device_kdriver_name
>   to make the API generic cross Linux and BSD, make it as EXPERIMENTAL.
> - Rebase on Zhihong's vDPA v3 patch set.
> - Minor code cleanup on vfio extension.
> 
> 
> Xiao Wang (5):
>   vfio: extend data structure for multi container
>   vfio: add multi container support
>   net/virtio: skip device probe in vdpa mode
>   net/ifcvf: add ifcvf vdpa driver
>   doc: add ifcvf driver document and release note


Hi Xiao,

Getting following build error for 32bit [1], can you please check them?

[1]
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘ifcvf_dma_map’:
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument
of type ‘long unsigned int’, but argument 6 has type ‘uint64_t {aka long long
unsigned int}’ [-Werror=format=]
   "%s(): " fmt "\n", __func__, ##args)
   ^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:22:
    "size 0x%lx.", i, reg->host_user_addr,
                      ~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’
   DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
   ^~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:37: note: format string is defined here
   DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
                                   ~~^
                                   %llx
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument
of type ‘long unsigned int’, but argument 7 has type ‘uint64_t {aka long long
unsigned int}’ [-Werror=format=]
   "%s(): " fmt "\n", __func__, ##args)
   ^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:4:
    reg->guest_phys_addr, reg->size);
    ~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’
   DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
   ^~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:48: note: format string is defined here
   DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
                                              ~~^
                                              %llx
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:24:3: error: format ‘%lx’ expects argument
of type ‘long unsigned int’, but argument 8 has type ‘uint64_t {aka long long
unsigned int}’ [-Werror=format=]
   "%s(): " fmt "\n", __func__, ##args)
   ^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:179:26:
    reg->guest_phys_addr, reg->size);
                          ~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:177:3: note: in expansion of macro ‘DRV_LOG’
   DRV_LOG(INFO, "region %u: HVA 0x%lx, GPA 0x%lx, "
   ^~~~~~~
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:178:14: note: format string is defined here
    "size 0x%lx.", i, reg->host_user_addr,
            ~~^
            %llx
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c: In function ‘vdpa_ifcvf_start’:
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:266:39: error: cast from pointer to
integer of different size [-Werror=pointer-to-int-cast]
   hw->vring[i].desc = qva_to_gpa(vid, (uint64_t)vq.desc);
                                       ^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:267:40: error: cast from pointer to
integer of different size [-Werror=pointer-to-int-cast]
   hw->vring[i].avail = qva_to_gpa(vid, (uint64_t)vq.avail);
                                        ^
.../dpdk/drivers/net/ifc/ifcvf_vdpa.c:268:39: error: cast from pointer to
integer of different size [-Werror=pointer-to-int-cast]
   hw->vring[i].used = qva_to_gpa(vid, (uint64_t)vq.used);
                                       ^