diff mbox

[dpdk-dev] eal: add option to force IOVA as PA mode

Message ID 20171130025739.4349-1-3chas3@gmail.com (mailing list archive)
State Rejected, archived
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Chas Williams Nov. 30, 2017, 2:57 a.m. UTC 
  From: Chas Williams <chas3@att.com>

The IOMMU in some machines report that they can only support
limited widths.  IOVA virtual addresses may exceed this width
making the use of IOVA virtual addresses difficult.  The option
CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical
address usage.

Signed-off-by: Chas Williams <chas3@att.com>
---
 config/common_base                | 5 +++++
 lib/librte_eal/bsdapp/eal/eal.c   | 4 ++++
 lib/librte_eal/linuxapp/eal/eal.c | 4 ++++
 3 files changed, 13 insertions(+)

Comments

Thomas Monjalon Nov. 30, 2017, 9:17 a.m. UTC | #1 
30/11/2017 03:57, Chas Williams:
> From: Chas Williams <chas3@att.com>
> 
> The IOMMU in some machines report that they can only support
> limited widths.  IOVA virtual addresses may exceed this width
> making the use of IOVA virtual addresses difficult.  The option
> CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical
> address usage.

Which machines are you talking about?
A run-time option may be a better solution.
Hemant Agrawal Dec. 1, 2017, 5:48 a.m. UTC | #2 
On 11/30/2017 8:27 AM, Chas Williams wrote:
> From: Chas Williams <chas3@att.com>
>
> The IOMMU in some machines report that they can only support
> limited widths.  IOVA virtual addresses may exceed this width
> making the use of IOVA virtual addresses difficult.  The option
> CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical
> address usage.
>

Is it possible to dynamically detect the machine limitation in bus iova 
code instead of compile time option?
Chas Williams Dec. 3, 2017, 12:23 a.m. UTC | #3 
The particular machine in my case is a Dell Optiplex 790 which is fairly
similar to another system that has basically the same problem.  The IOMMU
advertises that is can only handle 39 bits of addressing.  The DMAR address
tables have a width of 36 bits, so all is well when using IOVA physical
addresses.  With IOVA virtual addresses, they sometimes go beyond the 39
bit boundary.  I was under the impression that IOMMU widths are more
typically 48 bits, but I appear to have some low end systems with a simpler
IOMMU.

I could make it a runtime option.

On Thu, Nov 30, 2017 at 4:17 AM, Thomas Monjalon <thomas@monjalon.net>
wrote:

> 30/11/2017 03:57, Chas Williams:
> > From: Chas Williams <chas3@att.com>
> >
> > The IOMMU in some machines report that they can only support
> > limited widths.  IOVA virtual addresses may exceed this width
> > making the use of IOVA virtual addresses difficult.  The option
> > CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical
> > address usage.
>
> Which machines are you talking about?
> A run-time option may be a better solution.
>
>
Chas Williams Dec. 3, 2017, 12:25 a.m. UTC | #4 
On Fri, Dec 1, 2017 at 12:48 AM, Hemant Agrawal <hemant.agrawal@nxp.com>
wrote:

> On 11/30/2017 8:27 AM, Chas Williams wrote:
>
>> From: Chas Williams <chas3@att.com>
>>
>> The IOMMU in some machines report that they can only support
>> limited widths.  IOVA virtual addresses may exceed this width
>> making the use of IOVA virtual addresses difficult.  The option
>> CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical
>> address usage.
>>
>>
> Is it possible to dynamically detect the machine limitation in bus iova
> code instead of compile time option?
>
>
Yes, I think you can examine the "cap" of the IOMMU group in /sys and
determine the bit width.  I will have to check on that.
Thomas Monjalon Jan. 12, 2018, 12:12 a.m. UTC | #5 
03/12/2017 01:23, Chas Williams:
> The particular machine in my case is a Dell Optiplex 790 which is fairly
> similar to another system that has basically the same problem.  The IOMMU
> advertises that is can only handle 39 bits of addressing.  The DMAR address
> tables have a width of 36 bits, so all is well when using IOVA physical
> addresses.  With IOVA virtual addresses, they sometimes go beyond the 39
> bit boundary.  I was under the impression that IOMMU widths are more
> typically 48 bits, but I appear to have some low end systems with a simpler
> IOMMU.
> 
> I could make it a runtime option.
> 
> On Thu, Nov 30, 2017 at 4:17 AM, Thomas Monjalon <thomas@monjalon.net>
> wrote:
> 
> > 30/11/2017 03:57, Chas Williams:
> > > From: Chas Williams <chas3@att.com>
> > >
> > > The IOMMU in some machines report that they can only support
> > > limited widths.  IOVA virtual addresses may exceed this width
> > > making the use of IOVA virtual addresses difficult.  The option
> > > CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical
> > > address usage.
> >
> > Which machines are you talking about?
> > A run-time option may be a better solution.

Please check the patch from Maxime:
	https://dpdk.org/dev/patchwork/patch/33192/
Chas Williams Jan. 12, 2018, 3:28 a.m. UTC | #6 
Thanks, and yes I already saw that.  That seems like a much better
solution.  I will give it a spin in my box with this problem.

On Thu, Jan 11, 2018 at 7:12 PM, Thomas Monjalon <thomas@monjalon.net>
wrote:

> 03/12/2017 01:23, Chas Williams:
> > The particular machine in my case is a Dell Optiplex 790 which is fairly
> > similar to another system that has basically the same problem.  The IOMMU
> > advertises that is can only handle 39 bits of addressing.  The DMAR
> address
> > tables have a width of 36 bits, so all is well when using IOVA physical
> > addresses.  With IOVA virtual addresses, they sometimes go beyond the 39
> > bit boundary.  I was under the impression that IOMMU widths are more
> > typically 48 bits, but I appear to have some low end systems with a
> simpler
> > IOMMU.
> >
> > I could make it a runtime option.
> >
> > On Thu, Nov 30, 2017 at 4:17 AM, Thomas Monjalon <thomas@monjalon.net>
> > wrote:
> >
> > > 30/11/2017 03:57, Chas Williams:
> > > > From: Chas Williams <chas3@att.com>
> > > >
> > > > The IOMMU in some machines report that they can only support
> > > > limited widths.  IOVA virtual addresses may exceed this width
> > > > making the use of IOVA virtual addresses difficult.  The option
> > > > CONFIG_RTE_LIBRTE_USE_PHYS_IOVA can used to force IOVA physical
> > > > address usage.
> > >
> > > Which machines are you talking about?
> > > A run-time option may be a better solution.
>
> Please check the patch from Maxime:
>         https://dpdk.org/dev/patchwork/patch/33192/
>
>
>
diff mbox

Patch

diff --git a/config/common_base b/config/common_base
index e74febe..06b3ded 100644
--- a/config/common_base
+++ b/config/common_base
@@ -820,3 +820,8 @@  CONFIG_RTE_APP_CRYPTO_PERF=y
 # Compile the eventdev application
 #
 CONFIG_RTE_APP_EVENTDEV=y
+
+#
+# Force IOVA physical addresses in VFIO
+#
+CONFIG_RTE_LIBRTE_USE_PHYS_IOVA=n
diff --git a/lib/librte_eal/bsdapp/eal/eal.c b/lib/librte_eal/bsdapp/eal/eal.c
index 369a682..b396285 100644
--- a/lib/librte_eal/bsdapp/eal/eal.c
+++ b/lib/librte_eal/bsdapp/eal/eal.c
@@ -569,8 +569,12 @@  rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
+#ifdef RTE_LIBRTE_USE_PHYS_IOVA
+	rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA;
+#else
 	/* autodetect the iova mapping mode (default is iova_pa) */
 	rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class();
+#endif
 
 	if (internal_config.no_hugetlbfs == 0 &&
 			internal_config.process_type != RTE_PROC_SECONDARY &&
diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
index 229eec9..13b02fd 100644
--- a/lib/librte_eal/linuxapp/eal/eal.c
+++ b/lib/librte_eal/linuxapp/eal/eal.c
@@ -804,8 +804,12 @@  rte_eal_init(int argc, char **argv)
 		return -1;
 	}
 
+#ifdef RTE_LIBRTE_USE_PHYS_IOVA
+	rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA;
+#else
 	/* autodetect the iova mapping mode (default is iova_pa) */
 	rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class();
+#endif
 
 	/* Workaround for KNI which requires physical address to work */
 	if (rte_eal_get_configuration()->iova_mode == RTE_IOVA_VA &&