[dpdk-stable] [dpdk-dev] [PATCH v3] bus/pci: fix VF bus error for memory access

[Harman Kalra]

On Mon, Jun 22, 2020 at 07:13:51PM +0800, Haiyue Wang wrote:
> To fix CVE-2020-12888, the linux vfio-pci module will invalidate mmaps
> and block MMIO access on disabled memory, it will send a SIGBUS to the
> application:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__git.kernel.org_pub_scm_linux_kernel_git_torvalds_linux.git_commit_-3Fid-3Dabafbc551fddede3e0a08dee1dcde08fc0eb8476&d=DwIDAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=5ESHPj7V-7JdkxT_Z_SU6RrS37ys4UXudBQ_rrS5LRo&m=L4wvNYSLDBSsweJGHfGSw3cDNDi0ioJdsQbH2-pEVqE&s=9v3r9tYw6p5Paet2O3Nc2IPdOL1-o77RjYJx5H8G0vc&e= 
> 
> When the application opens the vfio PCI device, the vfio-pci module will
> enable the bus memory space through PCI read/write access. According to
> the PCIe specification, the 'Memory Space Enable' is always zero for VF:
> 
>              Table 9-13 Command Register Changes
> 
> Bit Location | PF and VF Register Differences | PF         | VF
>              | From Base                      | Attributes | Attributes
> -------------+--------------------------------+------------+-----------
>              | Memory Space Enable - Does not |            |
>              | apply to VFs. Must be hardwired|  Base      |  0b
>      1       | to 0b for VFs. VF Memory Space |            |
>              | is controlled by the VF MSE bit|            |
>              | in the VF Control register.    |            |
> -------------+--------------------------------+------------+-----------
> 
> Afterwards the vfio-pci will initialize its own virtual PCI config space
> data ('vconfig') by reading the VF's physical PCI config space, then the
> 'Memory Space Enable' bit in vconfig will always be 0b value. This will
> make the vfio-pci treat the BAR memory space as disabled, and the SIGBUS
> will be triggerred if access these BARs.
> 
> By investigation, the VF PCI device *passthrough* into the Guest OS by
> QEMU has the 'Memory Space Enable' with 1b value. That's because every
> PCI driver will start to enable the memory space, and this action will
> be hooked by vfio-pci virtual PCI read/write to set the 'Memory Space
> Enable' in vconfig space to 1b. So VF runs in guest OS has 'Mem+', but
> VF runs in host OS has 'Mem-'.
> 
> Align with PCI working mode in Guest/QEMU/Host, in DPDK, enable the PCI
> bus memory space explicitly to avoid access on disabled memory.
> 
> Fixes: 33604c31354a ("vfio: refactor PCI BAR mapping")
> Cc: stable at dpdk.org
> 
> Signed-off-by: Haiyue Wang <haiyue.wang at intel.com>

Tested-by: Harman Kalra <hkalra at marvell.com>

> ---
> v3: update the commit log, and fix one debug log with redundant
> description.
> v2: Rewrite the commit log, and put the link into it even it is long.
> ---
>  drivers/bus/pci/linux/pci_vfio.c | 37 ++++++++++++++++++++++++++++++++
>  1 file changed, 37 insertions(+)
> 
> diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c
> index 64cd84a68..ba60e7ce9 100644
> --- a/drivers/bus/pci/linux/pci_vfio.c
> +++ b/drivers/bus/pci/linux/pci_vfio.c
> @@ -149,6 +149,38 @@ pci_vfio_get_msix_bar(int fd, struct pci_msix_table *msix_table)
>  	return 0;
>  }
>  
> +/* enable PCI bus memory space */
> +static int
> +pci_vfio_enable_bus_memory(int dev_fd)
> +{
> +	uint16_t cmd;
> +	int ret;
> +
> +	ret = pread64(dev_fd, &cmd, sizeof(cmd),
> +		      VFIO_GET_REGION_ADDR(VFIO_PCI_CONFIG_REGION_INDEX) +
> +		      PCI_COMMAND);
> +
> +	if (ret != sizeof(cmd)) {
> +		RTE_LOG(ERR, EAL, "Cannot read command from PCI config space!\n");
> +		return -1;
> +	}
> +
> +	if (cmd & PCI_COMMAND_MEMORY)
> +		return 0;
> +
> +	cmd |= PCI_COMMAND_MEMORY;
> +	ret = pwrite64(dev_fd, &cmd, sizeof(cmd),
> +		       VFIO_GET_REGION_ADDR(VFIO_PCI_CONFIG_REGION_INDEX) +
> +		       PCI_COMMAND);
> +
> +	if (ret != sizeof(cmd)) {
> +		RTE_LOG(ERR, EAL, "Cannot write command to PCI config space!\n");
> +		return -1;
> +	}
> +
> +	return 0;
> +}
> +
>  /* set PCI bus mastering */
>  static int
>  pci_vfio_set_bus_master(int dev_fd, bool op)
> @@ -427,6 +459,11 @@ pci_rte_vfio_setup_device(struct rte_pci_device *dev, int vfio_dev_fd)
>  		return -1;
>  	}
>  
> +	if (pci_vfio_enable_bus_memory(vfio_dev_fd)) {
> +		RTE_LOG(ERR, EAL, "Cannot enable bus memory!\n");
> +		return -1;
> +	}
> +
>  	/* set bus mastering for the device */
>  	if (pci_vfio_set_bus_master(vfio_dev_fd, true)) {
>  		RTE_LOG(ERR, EAL, "Cannot set up bus mastering!\n");
> -- 
> 2.27.0
>