diff mbox

[dpdk-dev] eal: force IOVA PA mode if KNI module inserted

Message ID 3a029ac2-2b6b-b0c2-a577-967b6feba0c9@caviumnetworks.com (mailing list archive)
State Not Applicable, archived
Headers

Checks

Context Check Description
ci/checkpatch warning coding style issues
ci/Intel-compilation success Compilation OK

Commit Message

Santosh Shukla Nov. 2, 2017, 3:58 a.m. UTC 
  Hi Ferruh,


On Thursday 02 November 2017 05:36 AM, Ferruh Yigit wrote:
> Fix kernel crash with KNI because KNI requires physical addresses.
>
> When IOVA VA mode used, memzones and mbufs physical address fields
> contain virtual addresses. But KNI relies on these fields to enable
> kernel access for buffers. Those fields having virtual address cause
> crash in kernel.
>
> This is a workaround until KNI fixed properly to work with virtual
> addresses.
>
> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy")
>
> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
> ---
> Cc: Jianfeng Tan <jianfeng.tan@intel.com>
> Cc: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Cc: Santosh Shukla <santosh.shukla@caviumnetworks.com>
> Cc: Thomas Monjalon <thomas@monjalon.net>
>
> This patch superseded following one:
> http://dpdk.org/dev/patchwork/patch/31071/
> ---
>  lib/librte_eal/linuxapp/eal/eal.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
> index 017c402ed..29912a4e5 100644
> --- a/lib/librte_eal/linuxapp/eal/eal.c
> +++ b/lib/librte_eal/linuxapp/eal/eal.c
> @@ -808,6 +808,15 @@ rte_eal_init(int argc, char **argv)
>  	/* autodetect the iova mapping mode (default is iova_pa) */
>  	rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class();
>  
> +	/* Workaround for KNI which requires physical address to work */
> +	if (rte_eal_get_configuration()->iova_mode == RTE_IOVA_VA &&
> +			rte_eal_check_module("rte_kni") == 1) {
> +		rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA;
> +		RTE_LOG(WARNING, EAL,
> +			"Some devices want iova as va but pa will be used because.. "
> +			"KNI module inserted\n");
> +	}
> +
>  	if (internal_config.no_hugetlbfs == 0 &&
>  			internal_config.process_type != RTE_PROC_SECONDARY &&
>  			eal_hugepage_info_init() < 0) {

Rather checking for KNI module at linuxapp/eal, I was suggesting to move
KNI detection code in bus layer like below:

It builds but I haven;t tested, can you please give it a try.

Thanks.

Comments

Ferruh Yigit Nov. 2, 2017, 9:04 a.m. UTC | #1 
On 11/1/2017 8:58 PM, santosh wrote:
> Hi Ferruh,
> 
> 
> On Thursday 02 November 2017 05:36 AM, Ferruh Yigit wrote:
>> Fix kernel crash with KNI because KNI requires physical addresses.
>>
>> When IOVA VA mode used, memzones and mbufs physical address fields
>> contain virtual addresses. But KNI relies on these fields to enable
>> kernel access for buffers. Those fields having virtual address cause
>> crash in kernel.
>>
>> This is a workaround until KNI fixed properly to work with virtual
>> addresses.
>>
>> Fixes: 72d013644bd6 ("mem: honor IOVA mode in malloc virt2phy")
>>
>> Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
>> ---
>> Cc: Jianfeng Tan <jianfeng.tan@intel.com>
>> Cc: Jerin Jacob <jerin.jacob@caviumnetworks.com>
>> Cc: Santosh Shukla <santosh.shukla@caviumnetworks.com>
>> Cc: Thomas Monjalon <thomas@monjalon.net>
>>
>> This patch superseded following one:
>> http://dpdk.org/dev/patchwork/patch/31071/
>> ---
>>  lib/librte_eal/linuxapp/eal/eal.c | 9 +++++++++
>>  1 file changed, 9 insertions(+)
>>
>> diff --git a/lib/librte_eal/linuxapp/eal/eal.c b/lib/librte_eal/linuxapp/eal/eal.c
>> index 017c402ed..29912a4e5 100644
>> --- a/lib/librte_eal/linuxapp/eal/eal.c
>> +++ b/lib/librte_eal/linuxapp/eal/eal.c
>> @@ -808,6 +808,15 @@ rte_eal_init(int argc, char **argv)
>>  	/* autodetect the iova mapping mode (default is iova_pa) */
>>  	rte_eal_get_configuration()->iova_mode = rte_bus_get_iommu_class();
>>  
>> +	/* Workaround for KNI which requires physical address to work */
>> +	if (rte_eal_get_configuration()->iova_mode == RTE_IOVA_VA &&
>> +			rte_eal_check_module("rte_kni") == 1) {
>> +		rte_eal_get_configuration()->iova_mode = RTE_IOVA_PA;
>> +		RTE_LOG(WARNING, EAL,
>> +			"Some devices want iova as va but pa will be used because.. "
>> +			"KNI module inserted\n");
>> +	}
>> +
>>  	if (internal_config.no_hugetlbfs == 0 &&
>>  			internal_config.process_type != RTE_PROC_SECONDARY &&
>>  			eal_hugepage_info_init() < 0) {
> 
> Rather checking for KNI module at linuxapp/eal, I was suggesting to move
> KNI detection code in bus layer like below:

I was thinking same at first, but later was not sure about add this check into
bus related file, kni module check is so unrelated with pci.c .

Also this is not part of a regular process of detecting iova mode, which putting
into rte_pci_get_iommu_class() will look like it is. Instead this is a
workaround and we are overwriting the result of regular method for this special
case, so I think putting this into linux eal.c level after detection is good for
stressing the workaround.

So lets not spoil the logic in rte_pci_get_iommu_class() with kni module checks.

> 
> diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c
> index cdf8106..971586c 100644
> --- a/drivers/bus/pci/linux/pci.c
> +++ b/drivers/bus/pci/linux/pci.c
> @@ -586,11 +586,13 @@ rte_pci_get_iommu_class(void)
>         bool is_vfio_noiommu_enabled = true;
>         bool has_iova_va;
>         bool is_bound_uio;
> +       bool has_kni;
>  
>         is_bound = pci_one_device_is_bound();
>         if (!is_bound)
>                 return RTE_IOVA_DC;
>  
> +       has_kni = rte_eal_check_module("rte_kni");
>         has_iova_va = pci_one_device_has_iova_va();
>         is_bound_uio = pci_one_device_bound_uio();
>  #ifdef VFIO_PRESENT
> @@ -598,7 +600,7 @@ rte_pci_get_iommu_class(void)
>                                         true : false;
>  #endif
>  
> -       if (has_iova_va && !is_bound_uio && !is_vfio_noiommu_enabled)
> +       if (has_iova_va && !is_bound_uio && !has_kni && !is_vfio_noiommu_enabled)
>                 return RTE_IOVA_VA;
>  
>         if (has_iova_va) {
> @@ -607,6 +609,9 @@ rte_pci_get_iommu_class(void)
>                         RTE_LOG(WARNING, EAL, "vfio-noiommu mode configured\n");
>                 if (is_bound_uio)
>                         RTE_LOG(WARNING, EAL, "few device bound to UIO\n");
> +               if (has_kni)
> +                       RTE_LOG(WARNING, EAL, "Some devices want iova as va but pa will be used because.."
> +                                       "KNI module inserted\n");
>         }
>  
>         return RTE_IOVA_PA;
> It builds but I haven;t tested, can you please give it a try.
> 
> Thanks.
>
diff mbox

Patch

diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c
index cdf8106..971586c 100644
--- a/drivers/bus/pci/linux/pci.c
+++ b/drivers/bus/pci/linux/pci.c
@@ -586,11 +586,13 @@  rte_pci_get_iommu_class(void)
        bool is_vfio_noiommu_enabled = true;
        bool has_iova_va;
        bool is_bound_uio;
+       bool has_kni;
 
        is_bound = pci_one_device_is_bound();
        if (!is_bound)
                return RTE_IOVA_DC;
 
+       has_kni = rte_eal_check_module("rte_kni");
        has_iova_va = pci_one_device_has_iova_va();
        is_bound_uio = pci_one_device_bound_uio();
 #ifdef VFIO_PRESENT
@@ -598,7 +600,7 @@  rte_pci_get_iommu_class(void)
                                        true : false;
 #endif
 
-       if (has_iova_va && !is_bound_uio && !is_vfio_noiommu_enabled)
+       if (has_iova_va && !is_bound_uio && !has_kni && !is_vfio_noiommu_enabled)
                return RTE_IOVA_VA;
 
        if (has_iova_va) {
@@ -607,6 +609,9 @@  rte_pci_get_iommu_class(void)
                        RTE_LOG(WARNING, EAL, "vfio-noiommu mode configured\n");
                if (is_bound_uio)
                        RTE_LOG(WARNING, EAL, "few device bound to UIO\n");
+               if (has_kni)
+                       RTE_LOG(WARNING, EAL, "Some devices want iova as va but pa will be used because.."
+                                       "KNI module inserted\n");
        }
 
        return RTE_IOVA_PA;