[dpdk-dev,v4] bus/pci: fix vfio mode

Message ID 20171030223227.8852-1-ferruh.yigit@intel.com (mailing list archive)
State Accepted, archived
Headers

Checks

Context Check Description
ci/checkpatch warning coding style issues
ci/Intel-compilation success Compilation OK

Commit Message

Ferruh Yigit Oct. 30, 2017, 10:32 p.m. UTC
  Revert back to using VFIO_PRESENT as a marker to enable compilation
of VFIO-related segments.

VFIO_PRESENT is the combination of user configuration RTE_EAL_VFIO and
kernel version support check.

eal_vfio.h VFIO_PRESENT related check ordered to be compatible with
rte_vfio.h one, no functional modification.

Fixes: 279b581c897d ("vfio: expose functions")

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
v4:

* Alternative to v3, not superseding it, need to decide which one to
get. This one:
  * fixes vfio for new kernels
  * fixes build for old kernels for vfio enabled in config case
  * print warning if vfio enabled in config but not supported by kernel

* Independent from this patch, following may need fixing later
  * Kernel version check for vfio is broken for RTE_KERNELDIR
  * Kernel VFIO support check can be improved
---
 drivers/bus/pci/linux/pci_init.h             |  2 +-
 drivers/bus/pci/linux/pci_vfio.c             |  2 +-
 lib/librte_eal/common/include/rte_vfio.h     | 14 ++++++++++++++
 lib/librte_eal/linuxapp/eal/eal_interrupts.c |  1 -
 lib/librte_eal/linuxapp/eal/eal_vfio.h       | 14 ++++++++++----
 5 files changed, 26 insertions(+), 7 deletions(-)
  

Comments

Bruce Richardson Oct. 31, 2017, 12:03 p.m. UTC | #1
On Mon, Oct 30, 2017 at 10:32:27PM +0000, Ferruh Yigit wrote:
> Revert back to using VFIO_PRESENT as a marker to enable compilation
> of VFIO-related segments.
> 
> VFIO_PRESENT is the combination of user configuration RTE_EAL_VFIO and
> kernel version support check.
> 
> eal_vfio.h VFIO_PRESENT related check ordered to be compatible with
> rte_vfio.h one, no functional modification.
> 
> Fixes: 279b581c897d ("vfio: expose functions")
> 
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> v4:
> 
> * Alternative to v3, not superseding it, need to decide which one to
> get. This one:
>   * fixes vfio for new kernels
>   * fixes build for old kernels for vfio enabled in config case
>   * print warning if vfio enabled in config but not supported by kernel
> 
> * Independent from this patch, following may need fixing later
>   * Kernel version check for vfio is broken for RTE_KERNELDIR
>   * Kernel VFIO support check can be improved
> ---
>  drivers/bus/pci/linux/pci_init.h             |  2 +-
>  drivers/bus/pci/linux/pci_vfio.c             |  2 +-
>  lib/librte_eal/common/include/rte_vfio.h     | 14 ++++++++++++++
>  lib/librte_eal/linuxapp/eal/eal_interrupts.c |  1 -
>  lib/librte_eal/linuxapp/eal/eal_vfio.h       | 14 ++++++++++----
>  5 files changed, 26 insertions(+), 7 deletions(-)
> 
Hi,

just to confirm that applying this patch fixes DPDK on my system at
least.

From the discussion, I understood that this was a "compilation problem",
but in practice it manifests itself as compile working ok but your app
won't work with ports bound to vfio-pci. So please apply some fix for
this soon.

Thanks for the work on this, all.

Tested-by: Bruce Richardson <bruce.richardson@intel.com>
  
Gaëtan Rivet Oct. 31, 2017, 2:11 p.m. UTC | #2
Hi Ferruh,

On Mon, Oct 30, 2017 at 10:32:27PM +0000, Ferruh Yigit wrote:
> Revert back to using VFIO_PRESENT as a marker to enable compilation
> of VFIO-related segments.
> 
> VFIO_PRESENT is the combination of user configuration RTE_EAL_VFIO and
> kernel version support check.
> 
> eal_vfio.h VFIO_PRESENT related check ordered to be compatible with
> rte_vfio.h one, no functional modification.
> 
> Fixes: 279b581c897d ("vfio: expose functions")
> 
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> v4:
> 
> * Alternative to v3, not superseding it, need to decide which one to
> get. This one:
>   * fixes vfio for new kernels
>   * fixes build for old kernels for vfio enabled in config case
>   * print warning if vfio enabled in config but not supported by kernel
> 

Thanks for the v4.

Well, as I stated before, I am more of the opinion of letting the
VFIO_PRESENT define only within eal_vfio.h.

But in the end, as long as everything is working, I'm fine with it.
Unless someone is explicitly interested in going with my version, let's
keep yours. If needed I can always submit a new v4 that would actually
work with old kernels.

Sorry about the mess concerning VFIO. I found late in the work that I
needed to touch it, and did not have the test-bed for it.
  
Ferruh Yigit Oct. 31, 2017, 4:58 p.m. UTC | #3
On 10/31/2017 5:03 AM, Bruce Richardson wrote:
> On Mon, Oct 30, 2017 at 10:32:27PM +0000, Ferruh Yigit wrote:
>> Revert back to using VFIO_PRESENT as a marker to enable compilation
>> of VFIO-related segments.
>>
>> VFIO_PRESENT is the combination of user configuration RTE_EAL_VFIO and
>> kernel version support check.
>>
>> eal_vfio.h VFIO_PRESENT related check ordered to be compatible with
>> rte_vfio.h one, no functional modification.
>>
>> Fixes: 279b581c897d ("vfio: expose functions")
>>
>> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
>> ---
>> v4:
>>
>> * Alternative to v3, not superseding it, need to decide which one to
>> get. This one:
>>   * fixes vfio for new kernels
>>   * fixes build for old kernels for vfio enabled in config case
>>   * print warning if vfio enabled in config but not supported by kernel
>>
>> * Independent from this patch, following may need fixing later
>>   * Kernel version check for vfio is broken for RTE_KERNELDIR
>>   * Kernel VFIO support check can be improved
>> ---
>>  drivers/bus/pci/linux/pci_init.h             |  2 +-
>>  drivers/bus/pci/linux/pci_vfio.c             |  2 +-
>>  lib/librte_eal/common/include/rte_vfio.h     | 14 ++++++++++++++
>>  lib/librte_eal/linuxapp/eal/eal_interrupts.c |  1 -
>>  lib/librte_eal/linuxapp/eal/eal_vfio.h       | 14 ++++++++++----
>>  5 files changed, 26 insertions(+), 7 deletions(-)
>>
> Hi,
> 
> just to confirm that applying this patch fixes DPDK on my system at
> least.
> 
> From the discussion, I understood that this was a "compilation problem",
> but in practice it manifests itself as compile working ok but your app
> won't work with ports bound to vfio-pci. So please apply some fix for
> this soon.

The main reason of the patch is vfio issue. All three of patches from Jerin,
Gaetan, and this are fixing that issue.

Difference is:
- Create or not dependency to eal_vfio.h again
- Break or not build for old kernels

> 
> Thanks for the work on this, all.
> 
> Tested-by: Bruce Richardson <bruce.richardson@intel.com>
>
  
Thomas Monjalon Oct. 31, 2017, 6:21 p.m. UTC | #4
31/10/2017 13:03, Bruce Richardson:
> On Mon, Oct 30, 2017 at 10:32:27PM +0000, Ferruh Yigit wrote:
> > Revert back to using VFIO_PRESENT as a marker to enable compilation
> > of VFIO-related segments.
> > 
> > VFIO_PRESENT is the combination of user configuration RTE_EAL_VFIO and
> > kernel version support check.
> > 
> > eal_vfio.h VFIO_PRESENT related check ordered to be compatible with
> > rte_vfio.h one, no functional modification.
> > 
> > Fixes: 279b581c897d ("vfio: expose functions")
> > 
> > Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> > Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
> > Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> > ---
> > v4:
> > 
> > * Alternative to v3, not superseding it, need to decide which one to
> > get. This one:
> >   * fixes vfio for new kernels
> >   * fixes build for old kernels for vfio enabled in config case
> >   * print warning if vfio enabled in config but not supported by kernel
> > 
> > * Independent from this patch, following may need fixing later
> >   * Kernel version check for vfio is broken for RTE_KERNELDIR
> >   * Kernel VFIO support check can be improved
> > ---
> >  drivers/bus/pci/linux/pci_init.h             |  2 +-
> >  drivers/bus/pci/linux/pci_vfio.c             |  2 +-
> >  lib/librte_eal/common/include/rte_vfio.h     | 14 ++++++++++++++
> >  lib/librte_eal/linuxapp/eal/eal_interrupts.c |  1 -
> >  lib/librte_eal/linuxapp/eal/eal_vfio.h       | 14 ++++++++++----
> >  5 files changed, 26 insertions(+), 7 deletions(-)
> > 
> Hi,
> 
> just to confirm that applying this patch fixes DPDK on my system at
> least.
> 
> From the discussion, I understood that this was a "compilation problem",
> but in practice it manifests itself as compile working ok but your app
> won't work with ports bound to vfio-pci. So please apply some fix for
> this soon.
> 
> Thanks for the work on this, all.
> 
> Tested-by: Bruce Richardson <bruce.richardson@intel.com>

Applied, thanks
  

Patch

diff --git a/drivers/bus/pci/linux/pci_init.h b/drivers/bus/pci/linux/pci_init.h
index 99d7a2e86..f342c47de 100644
--- a/drivers/bus/pci/linux/pci_init.h
+++ b/drivers/bus/pci/linux/pci_init.h
@@ -74,7 +74,7 @@  void pci_uio_ioport_write(struct rte_pci_ioport *p,
 			  const void *data, size_t len, off_t offset);
 int pci_uio_ioport_unmap(struct rte_pci_ioport *p);
 
-#ifdef RTE_EAL_VFIO
+#ifdef VFIO_PRESENT
 
 #if LINUX_VERSION_CODE < KERNEL_VERSION(3, 10, 0)
 #define RTE_PCI_MSIX_TABLE_BIR    0x7
diff --git a/drivers/bus/pci/linux/pci_vfio.c b/drivers/bus/pci/linux/pci_vfio.c
index 360eed380..13d52a8b1 100644
--- a/drivers/bus/pci/linux/pci_vfio.c
+++ b/drivers/bus/pci/linux/pci_vfio.c
@@ -62,7 +62,7 @@ 
  * This file is only compiled if CONFIG_RTE_EAL_VFIO is set to "y".
  */
 
-#ifdef RTE_EAL_VFIO
+#ifdef VFIO_PRESENT
 
 #define PAGE_SIZE   (sysconf(_SC_PAGESIZE))
 #define PAGE_MASK   (~(PAGE_SIZE - 1))
diff --git a/lib/librte_eal/common/include/rte_vfio.h b/lib/librte_eal/common/include/rte_vfio.h
index a9b705741..d5934cdd3 100644
--- a/lib/librte_eal/common/include/rte_vfio.h
+++ b/lib/librte_eal/common/include/rte_vfio.h
@@ -34,6 +34,18 @@ 
 #ifndef _RTE_VFIO_H_
 #define _RTE_VFIO_H_
 
+/*
+ * determine if VFIO is present on the system
+ */
+#if !defined(VFIO_PRESENT) && defined(RTE_EAL_VFIO)
+#include <linux/version.h>
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 6, 0)
+#define VFIO_PRESENT
+#endif /* kernel version >= 3.6.0 */
+#endif /* RTE_EAL_VFIO */
+
+#ifdef VFIO_PRESENT
+
 #include <linux/vfio.h>
 
 #define VFIO_DIR "/dev/vfio"
@@ -136,4 +148,6 @@  int vfio_is_enabled(const char *modname);
  */
 int vfio_noiommu_is_enabled(void);
 
+#endif /* VFIO_PRESENT */
+
 #endif /* _RTE_VFIO_H_ */
diff --git a/lib/librte_eal/linuxapp/eal/eal_interrupts.c b/lib/librte_eal/linuxapp/eal/eal_interrupts.c
index cdd74ee10..0bebf004b 100644
--- a/lib/librte_eal/linuxapp/eal/eal_interrupts.c
+++ b/lib/librte_eal/linuxapp/eal/eal_interrupts.c
@@ -64,7 +64,6 @@ 
 #include <rte_errno.h>
 #include <rte_spinlock.h>
 #include <rte_pause.h>
-#include <rte_vfio.h>
 
 #include "eal_private.h"
 #include "eal_vfio.h"
diff --git a/lib/librte_eal/linuxapp/eal/eal_vfio.h b/lib/librte_eal/linuxapp/eal/eal_vfio.h
index 766d0042d..ba7892b72 100644
--- a/lib/librte_eal/linuxapp/eal/eal_vfio.h
+++ b/lib/librte_eal/linuxapp/eal/eal_vfio.h
@@ -37,9 +37,17 @@ 
 /*
  * determine if VFIO is present on the system
  */
-#ifdef RTE_EAL_VFIO
+#if !defined(VFIO_PRESENT) && defined(RTE_EAL_VFIO)
 #include <linux/version.h>
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 6, 0)
+#define VFIO_PRESENT
+#else
+#pragma message("VFIO configured but not supported by this kernel, disabling.")
+#endif /* kernel version >= 3.6.0 */
+#endif /* RTE_EAL_VFIO */
+
+#ifdef VFIO_PRESENT
+
 #include <linux/vfio.h>
 
 #define RTE_VFIO_TYPE1 VFIO_TYPE1_IOMMU
@@ -182,8 +190,6 @@  int vfio_mp_sync_setup(void);
 #define SOCKET_NO_FD 0x1
 #define SOCKET_ERR 0xFF
 
-#define VFIO_PRESENT
-#endif /* kernel version */
-#endif /* RTE_EAL_VFIO */
+#endif /* VFIO_PRESENT */
 
 #endif /* EAL_VFIO_H_ */