[v3,1/6] mem: add function for checking memsegs IOVAs addresses
Checks
Commit Message
A device can suffer addressing limitations. This functions checks
memsegs have iovas within the supported range based on dma mask.
PMD should use this during initialization if supported devices
suffer addressing limitations, returning an error if this function
returns memsegs out of range.
Another potential usage is for emulated IOMMU hardware with addressing
limitations.
Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
lib/librte_eal/common/eal_common_memory.c | 33 ++++++++++++++++++++++++++++++
lib/librte_eal/common/include/rte_memory.h | 3 +++
lib/librte_eal/rte_eal_version.map | 1 +
3 files changed, 37 insertions(+)
Comments
On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
> A device can suffer addressing limitations. This functions checks
> memsegs have iovas within the supported range based on dma mask.
>
> PMD should use this during initialization if supported devices
> suffer addressing limitations, returning an error if this function
> returns memsegs out of range.
>
> Another potential usage is for emulated IOMMU hardware with addressing
> limitations.
>
> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
> lib/librte_eal/common/eal_common_memory.c | 33
> ++++++++++++++++++++++++++++++
> lib/librte_eal/common/include/rte_memory.h | 3 +++
> lib/librte_eal/rte_eal_version.map | 1 +
> 3 files changed, 37 insertions(+)
>
> diff --git a/lib/librte_eal/common/eal_common_memory.c
> b/lib/librte_eal/common/eal_common_memory.c
> index fc6c44d..f5efebe 100644
> --- a/lib/librte_eal/common/eal_common_memory.c
> +++ b/lib/librte_eal/common/eal_common_memory.c
> @@ -109,6 +109,39 @@
> }
> }
>
> +/* check memseg iovas are within the required range based on dma mask
> */
> +int
> +rte_eal_check_dma_mask(uint8_t maskbits)
> +{
> +
> + const struct rte_mem_config *mcfg;
> + uint64_t mask;
> + int i;
> +
I think we should add some sanity check to the input maskbits, i.e.
[64,0) or [64, 32]? What would be a reasonable lower bound.
> + /* create dma mask */
> + mask = ~((1ULL << maskbits) - 1);
> +
> + /* get pointer to global configuration */
> + mcfg = rte_eal_get_configuration()->mem_config;
> +
> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
> + if (mcfg->memseg[i].addr == NULL)
> + break;
> +
> + if (mcfg->memseg[i].iova & mask) {
> + RTE_LOG(INFO, EAL,
> + "memseg[%d] iova %"PRIx64" out of range:\n",
> + i, mcfg->memseg[i].iova);
> +
> + RTE_LOG(INFO, EAL, "\tusing dma mask %"PRIx64"\n",
> + mask);
> + return -1;
> + }
> + }
> +
> + return 0;
> +}
> +
> /* return the number of memory channels */
> unsigned rte_memory_get_nchannel(void)
> {
> diff --git a/lib/librte_eal/common/include/rte_memory.h
> b/lib/librte_eal/common/include/rte_memory.h
> index 80a8fc0..b2a0168 100644
> --- a/lib/librte_eal/common/include/rte_memory.h
> +++ b/lib/librte_eal/common/include/rte_memory.h
> @@ -209,6 +209,9 @@ struct rte_memseg {
> */
> unsigned rte_memory_get_nrank(void);
>
> +/* check memsegs iovas are within a range based on dma mask */
> +int rte_eal_check_dma_mask(uint8_t maskbits);
> +
> /**
> * Drivers based on uio will not load unless physical
> * addresses are obtainable. It is only possible to get
> diff --git a/lib/librte_eal/rte_eal_version.map
> b/lib/librte_eal/rte_eal_version.map
> index f4f46c1..aa6cf87 100644
> --- a/lib/librte_eal/rte_eal_version.map
> +++ b/lib/librte_eal/rte_eal_version.map
> @@ -184,6 +184,7 @@ DPDK_17.11 {
>
> rte_eal_create_uio_dev;
> rte_bus_get_iommu_class;
> + rte_eal_check_dma_mask;
> rte_eal_has_pci;
> rte_eal_iova_mode;
> rte_eal_mbuf_default_mempool_ops;
> --
> 1.9.1
On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron <echaudro@redhat.com> wrote:
>
>
> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>
> A device can suffer addressing limitations. This functions checks
>> memsegs have iovas within the supported range based on dma mask.
>>
>> PMD should use this during initialization if supported devices
>> suffer addressing limitations, returning an error if this function
>> returns memsegs out of range.
>>
>> Another potential usage is for emulated IOMMU hardware with addressing
>> limitations.
>>
>> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>> lib/librte_eal/common/eal_common_memory.c | 33
>> ++++++++++++++++++++++++++++++
>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>> lib/librte_eal/rte_eal_version.map | 1 +
>> 3 files changed, 37 insertions(+)
>>
>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>> b/lib/librte_eal/common/eal_common_memory.c
>> index fc6c44d..f5efebe 100644
>> --- a/lib/librte_eal/common/eal_common_memory.c
>> +++ b/lib/librte_eal/common/eal_common_memory.c
>> @@ -109,6 +109,39 @@
>> }
>> }
>>
>> +/* check memseg iovas are within the required range based on dma mask */
>> +int
>> +rte_eal_check_dma_mask(uint8_t maskbits)
>> +{
>> +
>> + const struct rte_mem_config *mcfg;
>> + uint64_t mask;
>> + int i;
>> +
>>
>
> I think we should add some sanity check to the input maskbits, i.e. [64,0)
> or [64, 32]? What would be a reasonable lower bound.
>
>
This is not a user's API, so any invocation will be reviewed, but I guess
adding a sanity check here does not harm.
Not sure about lower bound but upper should 64, although it does not make
sense but it is safe. Lower bound is not so problematic.
>
> + /* create dma mask */
>> + mask = ~((1ULL << maskbits) - 1);
>> +
>> + /* get pointer to global configuration */
>> + mcfg = rte_eal_get_configuration()->mem_config;
>> +
>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>> + if (mcfg->memseg[i].addr == NULL)
>> + break;
>> +
>> + if (mcfg->memseg[i].iova & mask) {
>> + RTE_LOG(INFO, EAL,
>> + "memseg[%d] iova %"PRIx64" out of
>> range:\n",
>> + i, mcfg->memseg[i].iova);
>> +
>> + RTE_LOG(INFO, EAL, "\tusing dma mask %"PRIx64"\n",
>> + mask);
>> + return -1;
>> + }
>> + }
>> +
>> + return 0;
>> +}
>> +
>> /* return the number of memory channels */
>> unsigned rte_memory_get_nchannel(void)
>> {
>> diff --git a/lib/librte_eal/common/include/rte_memory.h
>> b/lib/librte_eal/common/include/rte_memory.h
>> index 80a8fc0..b2a0168 100644
>> --- a/lib/librte_eal/common/include/rte_memory.h
>> +++ b/lib/librte_eal/common/include/rte_memory.h
>> @@ -209,6 +209,9 @@ struct rte_memseg {
>> */
>> unsigned rte_memory_get_nrank(void);
>>
>> +/* check memsegs iovas are within a range based on dma mask */
>> +int rte_eal_check_dma_mask(uint8_t maskbits);
>> +
>> /**
>> * Drivers based on uio will not load unless physical
>> * addresses are obtainable. It is only possible to get
>> diff --git a/lib/librte_eal/rte_eal_version.map
>> b/lib/librte_eal/rte_eal_version.map
>> index f4f46c1..aa6cf87 100644
>> --- a/lib/librte_eal/rte_eal_version.map
>> +++ b/lib/librte_eal/rte_eal_version.map
>> @@ -184,6 +184,7 @@ DPDK_17.11 {
>>
>> rte_eal_create_uio_dev;
>> rte_bus_get_iommu_class;
>> + rte_eal_check_dma_mask;
>> rte_eal_has_pci;
>> rte_eal_iova_mode;
>> rte_eal_mbuf_default_mempool_ops;
>> --
>> 1.9.1
>>
>
On 10 Jul 2018, at 11:34, Alejandro Lucero wrote:
> On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron <echaudro@redhat.com>
> wrote:
>
>>
>>
>> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>>
>> A device can suffer addressing limitations. This functions checks
>>> memsegs have iovas within the supported range based on dma mask.
>>>
>>> PMD should use this during initialization if supported devices
>>> suffer addressing limitations, returning an error if this function
>>> returns memsegs out of range.
>>>
>>> Another potential usage is for emulated IOMMU hardware with
>>> addressing
>>> limitations.
>>>
>>> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
>>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>> ---
>>> lib/librte_eal/common/eal_common_memory.c | 33
>>> ++++++++++++++++++++++++++++++
>>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>>> lib/librte_eal/rte_eal_version.map | 1 +
>>> 3 files changed, 37 insertions(+)
>>>
>>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>>> b/lib/librte_eal/common/eal_common_memory.c
>>> index fc6c44d..f5efebe 100644
>>> --- a/lib/librte_eal/common/eal_common_memory.c
>>> +++ b/lib/librte_eal/common/eal_common_memory.c
>>> @@ -109,6 +109,39 @@
>>> }
>>> }
>>>
>>> +/* check memseg iovas are within the required range based on dma
>>> mask */
>>> +int
>>> +rte_eal_check_dma_mask(uint8_t maskbits)
>>> +{
>>> +
>>> + const struct rte_mem_config *mcfg;
>>> + uint64_t mask;
>>> + int i;
>>> +
>>>
>>
>> I think we should add some sanity check to the input maskbits, i.e.
>> [64,0)
>> or [64, 32]? What would be a reasonable lower bound.
>>
>>
> This is not a user's API, so any invocation will be reviewed, but I
> guess
> adding a sanity check here does not harm.
>
> Not sure about lower bound but upper should 64, although it does not
> make
> sense but it is safe. Lower bound is not so problematic.
>
>
>>
>> + /* create dma mask */
>>> + mask = ~((1ULL << maskbits) - 1);
>>> +
>>> + /* get pointer to global configuration */
>>> + mcfg = rte_eal_get_configuration()->mem_config;
>>> +
>>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>>> + if (mcfg->memseg[i].addr == NULL)
>>> + break;
Looking at some other code, it looks like NULL entries might exists. So
should a continue; rather than a break; be used here?
>>> +
>>> + if (mcfg->memseg[i].iova & mask) {
>>> + RTE_LOG(INFO, EAL,
>>> + "memseg[%d] iova %"PRIx64" out of
>>> range:\n",
>>> + i, mcfg->memseg[i].iova);
>>> +
>>> + RTE_LOG(INFO, EAL, "\tusing dma mask
>>> %"PRIx64"\n",
>>> + mask);
>>> + return -1;
>>> + }
>>> + }
>>> +
>>> + return 0;
>>> +}
>>> +
>>> /* return the number of memory channels */
>>> unsigned rte_memory_get_nchannel(void)
>>> {
>>> diff --git a/lib/librte_eal/common/include/rte_memory.h
>>> b/lib/librte_eal/common/include/rte_memory.h
>>> index 80a8fc0..b2a0168 100644
>>> --- a/lib/librte_eal/common/include/rte_memory.h
>>> +++ b/lib/librte_eal/common/include/rte_memory.h
>>> @@ -209,6 +209,9 @@ struct rte_memseg {
>>> */
>>> unsigned rte_memory_get_nrank(void);
>>>
>>> +/* check memsegs iovas are within a range based on dma mask */
>>> +int rte_eal_check_dma_mask(uint8_t maskbits);
>>> +
>>> /**
>>> * Drivers based on uio will not load unless physical
>>> * addresses are obtainable. It is only possible to get
>>> diff --git a/lib/librte_eal/rte_eal_version.map
>>> b/lib/librte_eal/rte_eal_version.map
>>> index f4f46c1..aa6cf87 100644
>>> --- a/lib/librte_eal/rte_eal_version.map
>>> +++ b/lib/librte_eal/rte_eal_version.map
>>> @@ -184,6 +184,7 @@ DPDK_17.11 {
>>>
>>> rte_eal_create_uio_dev;
>>> rte_bus_get_iommu_class;
>>> + rte_eal_check_dma_mask;
>>> rte_eal_has_pci;
>>> rte_eal_iova_mode;
>>> rte_eal_mbuf_default_mempool_ops;
>>> --
>>> 1.9.1
>>>
>>
On Tue, Jul 10, 2018 at 11:06 AM, Eelco Chaudron <echaudro@redhat.com>
wrote:
>
>
> On 10 Jul 2018, at 11:34, Alejandro Lucero wrote:
>
> On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron <echaudro@redhat.com>
>> wrote:
>>
>>
>>>
>>> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>>>
>>> A device can suffer addressing limitations. This functions checks
>>>
>>>> memsegs have iovas within the supported range based on dma mask.
>>>>
>>>> PMD should use this during initialization if supported devices
>>>> suffer addressing limitations, returning an error if this function
>>>> returns memsegs out of range.
>>>>
>>>> Another potential usage is for emulated IOMMU hardware with addressing
>>>> limitations.
>>>>
>>>> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
>>>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>> ---
>>>> lib/librte_eal/common/eal_common_memory.c | 33
>>>> ++++++++++++++++++++++++++++++
>>>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>>>> lib/librte_eal/rte_eal_version.map | 1 +
>>>> 3 files changed, 37 insertions(+)
>>>>
>>>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>>>> b/lib/librte_eal/common/eal_common_memory.c
>>>> index fc6c44d..f5efebe 100644
>>>> --- a/lib/librte_eal/common/eal_common_memory.c
>>>> +++ b/lib/librte_eal/common/eal_common_memory.c
>>>> @@ -109,6 +109,39 @@
>>>> }
>>>> }
>>>>
>>>> +/* check memseg iovas are within the required range based on dma mask
>>>> */
>>>> +int
>>>> +rte_eal_check_dma_mask(uint8_t maskbits)
>>>> +{
>>>> +
>>>> + const struct rte_mem_config *mcfg;
>>>> + uint64_t mask;
>>>> + int i;
>>>> +
>>>>
>>>>
>>> I think we should add some sanity check to the input maskbits, i.e.
>>> [64,0)
>>> or [64, 32]? What would be a reasonable lower bound.
>>>
>>>
>>> This is not a user's API, so any invocation will be reviewed, but I guess
>> adding a sanity check here does not harm.
>>
>> Not sure about lower bound but upper should 64, although it does not make
>> sense but it is safe. Lower bound is not so problematic.
>>
>>
>>
>>> + /* create dma mask */
>>>
>>>> + mask = ~((1ULL << maskbits) - 1);
>>>> +
>>>> + /* get pointer to global configuration */
>>>> + mcfg = rte_eal_get_configuration()->mem_config;
>>>> +
>>>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>>>> + if (mcfg->memseg[i].addr == NULL)
>>>> + break;
>>>>
>>>
> Looking at some other code, it looks like NULL entries might exists. So
> should a continue; rather than a break; be used here?
>
>
I do not think so. memsegs are allocated sequentially, so first with addr
as NULL implies no more memsegs.
>
> +
>>>> + if (mcfg->memseg[i].iova & mask) {
>>>> + RTE_LOG(INFO, EAL,
>>>> + "memseg[%d] iova %"PRIx64" out of
>>>> range:\n",
>>>> + i, mcfg->memseg[i].iova);
>>>> +
>>>> + RTE_LOG(INFO, EAL, "\tusing dma mask
>>>> %"PRIx64"\n",
>>>> + mask);
>>>> + return -1;
>>>> + }
>>>> + }
>>>> +
>>>> + return 0;
>>>> +}
>>>> +
>>>> /* return the number of memory channels */
>>>> unsigned rte_memory_get_nchannel(void)
>>>> {
>>>> diff --git a/lib/librte_eal/common/include/rte_memory.h
>>>> b/lib/librte_eal/common/include/rte_memory.h
>>>> index 80a8fc0..b2a0168 100644
>>>> --- a/lib/librte_eal/common/include/rte_memory.h
>>>> +++ b/lib/librte_eal/common/include/rte_memory.h
>>>> @@ -209,6 +209,9 @@ struct rte_memseg {
>>>> */
>>>> unsigned rte_memory_get_nrank(void);
>>>>
>>>> +/* check memsegs iovas are within a range based on dma mask */
>>>> +int rte_eal_check_dma_mask(uint8_t maskbits);
>>>> +
>>>> /**
>>>> * Drivers based on uio will not load unless physical
>>>> * addresses are obtainable. It is only possible to get
>>>> diff --git a/lib/librte_eal/rte_eal_version.map
>>>> b/lib/librte_eal/rte_eal_version.map
>>>> index f4f46c1..aa6cf87 100644
>>>> --- a/lib/librte_eal/rte_eal_version.map
>>>> +++ b/lib/librte_eal/rte_eal_version.map
>>>> @@ -184,6 +184,7 @@ DPDK_17.11 {
>>>>
>>>> rte_eal_create_uio_dev;
>>>> rte_bus_get_iommu_class;
>>>> + rte_eal_check_dma_mask;
>>>> rte_eal_has_pci;
>>>> rte_eal_iova_mode;
>>>> rte_eal_mbuf_default_mempool_ops;
>>>> --
>>>> 1.9.1
>>>>
>>>>
>>>
>
>
On 10 Jul 2018, at 12:52, Alejandro Lucero wrote:
> On Tue, Jul 10, 2018 at 11:06 AM, Eelco Chaudron <echaudro@redhat.com>
> wrote:
>
>>
>>
>> On 10 Jul 2018, at 11:34, Alejandro Lucero wrote:
>>
>> On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron <echaudro@redhat.com>
>>> wrote:
>>>
>>>
>>>>
>>>> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>>>>
>>>> A device can suffer addressing limitations. This functions checks
>>>>
>>>>> memsegs have iovas within the supported range based on dma mask.
>>>>>
>>>>> PMD should use this during initialization if supported devices
>>>>> suffer addressing limitations, returning an error if this function
>>>>> returns memsegs out of range.
>>>>>
>>>>> Another potential usage is for emulated IOMMU hardware with
>>>>> addressing
>>>>> limitations.
>>>>>
>>>>> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
>>>>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>>> ---
>>>>> lib/librte_eal/common/eal_common_memory.c | 33
>>>>> ++++++++++++++++++++++++++++++
>>>>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>>>>> lib/librte_eal/rte_eal_version.map | 1 +
>>>>> 3 files changed, 37 insertions(+)
>>>>>
>>>>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>>>>> b/lib/librte_eal/common/eal_common_memory.c
>>>>> index fc6c44d..f5efebe 100644
>>>>> --- a/lib/librte_eal/common/eal_common_memory.c
>>>>> +++ b/lib/librte_eal/common/eal_common_memory.c
>>>>> @@ -109,6 +109,39 @@
>>>>> }
>>>>> }
>>>>>
>>>>> +/* check memseg iovas are within the required range based on dma
>>>>> mask
>>>>> */
>>>>> +int
>>>>> +rte_eal_check_dma_mask(uint8_t maskbits)
>>>>> +{
>>>>> +
>>>>> + const struct rte_mem_config *mcfg;
>>>>> + uint64_t mask;
>>>>> + int i;
>>>>> +
>>>>>
>>>>>
>>>> I think we should add some sanity check to the input maskbits, i.e.
>>>> [64,0)
>>>> or [64, 32]? What would be a reasonable lower bound.
>>>>
>>>>
>>>> This is not a user's API, so any invocation will be reviewed, but I
>>>> guess
>>> adding a sanity check here does not harm.
>>>
>>> Not sure about lower bound but upper should 64, although it does not
>>> make
>>> sense but it is safe. Lower bound is not so problematic.
>>>
>>>
>>>
>>>> + /* create dma mask */
>>>>
>>>>> + mask = ~((1ULL << maskbits) - 1);
>>>>> +
>>>>> + /* get pointer to global configuration */
>>>>> + mcfg = rte_eal_get_configuration()->mem_config;
>>>>> +
>>>>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>>>>> + if (mcfg->memseg[i].addr == NULL)
>>>>> + break;
>>>>>
>>>>
>> Looking at some other code, it looks like NULL entries might exists.
>> So
>> should a continue; rather than a break; be used here?
>>
>>
> I do not think so. memsegs are allocated sequentially, so first with
> addr
> as NULL implies no more memsegs.
I was referring to the mem walk functions, rte_memseg_list_walk(). Maybe
some having more experience with this area can review/comment.
>
>
>>
>> +
>>>>> + if (mcfg->memseg[i].iova & mask) {
>>>>> + RTE_LOG(INFO, EAL,
>>>>> + "memseg[%d] iova %"PRIx64" out of
>>>>> range:\n",
>>>>> + i, mcfg->memseg[i].iova);
>>>>> +
>>>>> + RTE_LOG(INFO, EAL, "\tusing dma mask
>>>>> %"PRIx64"\n",
>>>>> + mask);
>>>>> + return -1;
>>>>> + }
>>>>> + }
>>>>> +
>>>>> + return 0;
>>>>> +}
>>>>> +
>>>>> /* return the number of memory channels */
>>>>> unsigned rte_memory_get_nchannel(void)
>>>>> {
>>>>> diff --git a/lib/librte_eal/common/include/rte_memory.h
>>>>> b/lib/librte_eal/common/include/rte_memory.h
>>>>> index 80a8fc0..b2a0168 100644
>>>>> --- a/lib/librte_eal/common/include/rte_memory.h
>>>>> +++ b/lib/librte_eal/common/include/rte_memory.h
>>>>> @@ -209,6 +209,9 @@ struct rte_memseg {
>>>>> */
>>>>> unsigned rte_memory_get_nrank(void);
>>>>>
>>>>> +/* check memsegs iovas are within a range based on dma mask */
>>>>> +int rte_eal_check_dma_mask(uint8_t maskbits);
>>>>> +
>>>>> /**
>>>>> * Drivers based on uio will not load unless physical
>>>>> * addresses are obtainable. It is only possible to get
>>>>> diff --git a/lib/librte_eal/rte_eal_version.map
>>>>> b/lib/librte_eal/rte_eal_version.map
>>>>> index f4f46c1..aa6cf87 100644
>>>>> --- a/lib/librte_eal/rte_eal_version.map
>>>>> +++ b/lib/librte_eal/rte_eal_version.map
>>>>> @@ -184,6 +184,7 @@ DPDK_17.11 {
>>>>>
>>>>> rte_eal_create_uio_dev;
>>>>> rte_bus_get_iommu_class;
>>>>> + rte_eal_check_dma_mask;
>>>>> rte_eal_has_pci;
>>>>> rte_eal_iova_mode;
>>>>> rte_eal_mbuf_default_mempool_ops;
>>>>> --
>>>>> 1.9.1
>>>>>
>>>>>
>>>>
>>
>>
On 10-Jul-18 12:14 PM, Eelco Chaudron wrote:
>
>
> On 10 Jul 2018, at 12:52, Alejandro Lucero wrote:
>
>> On Tue, Jul 10, 2018 at 11:06 AM, Eelco Chaudron <echaudro@redhat.com>
>> wrote:
>>
>>>
>>>
>>> On 10 Jul 2018, at 11:34, Alejandro Lucero wrote:
>>>
>>> On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron <echaudro@redhat.com>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>>>>>
>>>>> A device can suffer addressing limitations. This functions checks
>>>>>
>>>>>> memsegs have iovas within the supported range based on dma mask.
>>>>>>
>>>>>> PMD should use this during initialization if supported devices
>>>>>> suffer addressing limitations, returning an error if this function
>>>>>> returns memsegs out of range.
>>>>>>
>>>>>> Another potential usage is for emulated IOMMU hardware with
>>>>>> addressing
>>>>>> limitations.
>>>>>>
>>>>>> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
>>>>>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>>>> ---
>>>>>> lib/librte_eal/common/eal_common_memory.c | 33
>>>>>> ++++++++++++++++++++++++++++++
>>>>>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>>>>>> lib/librte_eal/rte_eal_version.map | 1 +
>>>>>> 3 files changed, 37 insertions(+)
>>>>>>
>>>>>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>>>>>> b/lib/librte_eal/common/eal_common_memory.c
>>>>>> index fc6c44d..f5efebe 100644
>>>>>> --- a/lib/librte_eal/common/eal_common_memory.c
>>>>>> +++ b/lib/librte_eal/common/eal_common_memory.c
>>>>>> @@ -109,6 +109,39 @@
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> +/* check memseg iovas are within the required range based on dma
>>>>>> mask
>>>>>> */
>>>>>> +int
>>>>>> +rte_eal_check_dma_mask(uint8_t maskbits)
>>>>>> +{
>>>>>> +
>>>>>> + const struct rte_mem_config *mcfg;
>>>>>> + uint64_t mask;
>>>>>> + int i;
>>>>>> +
>>>>>>
>>>>>>
>>>>> I think we should add some sanity check to the input maskbits, i.e.
>>>>> [64,0)
>>>>> or [64, 32]? What would be a reasonable lower bound.
>>>>>
>>>>>
>>>>> This is not a user's API, so any invocation will be reviewed, but I
>>>>> guess
>>>> adding a sanity check here does not harm.
>>>>
>>>> Not sure about lower bound but upper should 64, although it does not
>>>> make
>>>> sense but it is safe. Lower bound is not so problematic.
>>>>
>>>>
>>>>
>>>>> + /* create dma mask */
>>>>>
>>>>>> + mask = ~((1ULL << maskbits) - 1);
>>>>>> +
>>>>>> + /* get pointer to global configuration */
>>>>>> + mcfg = rte_eal_get_configuration()->mem_config;
>>>>>> +
>>>>>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>>>>>> + if (mcfg->memseg[i].addr == NULL)
>>>>>> + break;
>>>>>>
>>>>>
>>> Looking at some other code, it looks like NULL entries might exists. So
>>> should a continue; rather than a break; be used here?
>>>
>>>
>> I do not think so. memsegs are allocated sequentially, so first with addr
>> as NULL implies no more memsegs.
>
> I was referring to the mem walk functions, rte_memseg_list_walk(). Maybe
> some having more experience with this area can review/comment.
Pre-18.05, all memsegs are allocated continuously. Memseg lists and
memseg list walk functions are 18.05+.
Alejandro, perhaps it would be worth it to tag your patchset with
"pre-18.05" to avoid similar confusion in the future?
On Tue, Jul 10, 2018 at 12:14 PM, Eelco Chaudron <echaudro@redhat.com>
wrote:
>
>
> On 10 Jul 2018, at 12:52, Alejandro Lucero wrote:
>
> On Tue, Jul 10, 2018 at 11:06 AM, Eelco Chaudron <echaudro@redhat.com>
>> wrote:
>>
>>
>>>
>>> On 10 Jul 2018, at 11:34, Alejandro Lucero wrote:
>>>
>>> On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron <echaudro@redhat.com>
>>>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>>>>>
>>>>> A device can suffer addressing limitations. This functions checks
>>>>>
>>>>> memsegs have iovas within the supported range based on dma mask.
>>>>>>
>>>>>> PMD should use this during initialization if supported devices
>>>>>> suffer addressing limitations, returning an error if this function
>>>>>> returns memsegs out of range.
>>>>>>
>>>>>> Another potential usage is for emulated IOMMU hardware with addressing
>>>>>> limitations.
>>>>>>
>>>>>> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
>>>>>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>>>> ---
>>>>>> lib/librte_eal/common/eal_common_memory.c | 33
>>>>>> ++++++++++++++++++++++++++++++
>>>>>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>>>>>> lib/librte_eal/rte_eal_version.map | 1 +
>>>>>> 3 files changed, 37 insertions(+)
>>>>>>
>>>>>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>>>>>> b/lib/librte_eal/common/eal_common_memory.c
>>>>>> index fc6c44d..f5efebe 100644
>>>>>> --- a/lib/librte_eal/common/eal_common_memory.c
>>>>>> +++ b/lib/librte_eal/common/eal_common_memory.c
>>>>>> @@ -109,6 +109,39 @@
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> +/* check memseg iovas are within the required range based on dma mask
>>>>>> */
>>>>>> +int
>>>>>> +rte_eal_check_dma_mask(uint8_t maskbits)
>>>>>> +{
>>>>>> +
>>>>>> + const struct rte_mem_config *mcfg;
>>>>>> + uint64_t mask;
>>>>>> + int i;
>>>>>> +
>>>>>>
>>>>>>
>>>>>> I think we should add some sanity check to the input maskbits, i.e.
>>>>> [64,0)
>>>>> or [64, 32]? What would be a reasonable lower bound.
>>>>>
>>>>>
>>>>> This is not a user's API, so any invocation will be reviewed, but I
>>>>> guess
>>>>>
>>>> adding a sanity check here does not harm.
>>>>
>>>> Not sure about lower bound but upper should 64, although it does not
>>>> make
>>>> sense but it is safe. Lower bound is not so problematic.
>>>>
>>>>
>>>>
>>>> + /* create dma mask */
>>>>>
>>>>> + mask = ~((1ULL << maskbits) - 1);
>>>>>> +
>>>>>> + /* get pointer to global configuration */
>>>>>> + mcfg = rte_eal_get_configuration()->mem_config;
>>>>>> +
>>>>>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>>>>>> + if (mcfg->memseg[i].addr == NULL)
>>>>>> + break;
>>>>>>
>>>>>>
>>>>> Looking at some other code, it looks like NULL entries might exists. So
>>> should a continue; rather than a break; be used here?
>>>
>>>
>>> I do not think so. memsegs are allocated sequentially, so first with addr
>> as NULL implies no more memsegs.
>>
>
> I was referring to the mem walk functions, rte_memseg_list_walk(). Maybe
> some having more experience with this area can review/comment.
>
>
This patchset applies to 17.11.3 which has not that function implemented.
You can see what rte_eal_get_physmem_size and rte_dump_physmem_layout do in
lib/librte_eal/common/eal_common_memory.c file regarding memseg "walks"
when addr is NULL.
>
>
>>
>>
>>> +
>>>
>>>> + if (mcfg->memseg[i].iova & mask) {
>>>>>> + RTE_LOG(INFO, EAL,
>>>>>> + "memseg[%d] iova %"PRIx64" out of
>>>>>> range:\n",
>>>>>> + i, mcfg->memseg[i].iova);
>>>>>> +
>>>>>> + RTE_LOG(INFO, EAL, "\tusing dma mask
>>>>>> %"PRIx64"\n",
>>>>>> + mask);
>>>>>> + return -1;
>>>>>> + }
>>>>>> + }
>>>>>> +
>>>>>> + return 0;
>>>>>> +}
>>>>>> +
>>>>>> /* return the number of memory channels */
>>>>>> unsigned rte_memory_get_nchannel(void)
>>>>>> {
>>>>>> diff --git a/lib/librte_eal/common/include/rte_memory.h
>>>>>> b/lib/librte_eal/common/include/rte_memory.h
>>>>>> index 80a8fc0..b2a0168 100644
>>>>>> --- a/lib/librte_eal/common/include/rte_memory.h
>>>>>> +++ b/lib/librte_eal/common/include/rte_memory.h
>>>>>> @@ -209,6 +209,9 @@ struct rte_memseg {
>>>>>> */
>>>>>> unsigned rte_memory_get_nrank(void);
>>>>>>
>>>>>> +/* check memsegs iovas are within a range based on dma mask */
>>>>>> +int rte_eal_check_dma_mask(uint8_t maskbits);
>>>>>> +
>>>>>> /**
>>>>>> * Drivers based on uio will not load unless physical
>>>>>> * addresses are obtainable. It is only possible to get
>>>>>> diff --git a/lib/librte_eal/rte_eal_version.map
>>>>>> b/lib/librte_eal/rte_eal_version.map
>>>>>> index f4f46c1..aa6cf87 100644
>>>>>> --- a/lib/librte_eal/rte_eal_version.map
>>>>>> +++ b/lib/librte_eal/rte_eal_version.map
>>>>>> @@ -184,6 +184,7 @@ DPDK_17.11 {
>>>>>>
>>>>>> rte_eal_create_uio_dev;
>>>>>> rte_bus_get_iommu_class;
>>>>>> + rte_eal_check_dma_mask;
>>>>>> rte_eal_has_pci;
>>>>>> rte_eal_iova_mode;
>>>>>> rte_eal_mbuf_default_mempool_ops;
>>>>>> --
>>>>>> 1.9.1
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>
>
On Tue, Jul 10, 2018 at 12:33 PM, Burakov, Anatoly <
anatoly.burakov@intel.com> wrote:
> On 10-Jul-18 12:14 PM, Eelco Chaudron wrote:
>
>>
>>
>> On 10 Jul 2018, at 12:52, Alejandro Lucero wrote:
>>
>> On Tue, Jul 10, 2018 at 11:06 AM, Eelco Chaudron <echaudro@redhat.com>
>>> wrote:
>>>
>>>
>>>>
>>>> On 10 Jul 2018, at 11:34, Alejandro Lucero wrote:
>>>>
>>>> On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron <echaudro@redhat.com>
>>>>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>>>>>>
>>>>>> A device can suffer addressing limitations. This functions checks
>>>>>>
>>>>>> memsegs have iovas within the supported range based on dma mask.
>>>>>>>
>>>>>>> PMD should use this during initialization if supported devices
>>>>>>> suffer addressing limitations, returning an error if this function
>>>>>>> returns memsegs out of range.
>>>>>>>
>>>>>>> Another potential usage is for emulated IOMMU hardware with
>>>>>>> addressing
>>>>>>> limitations.
>>>>>>>
>>>>>>> Signed-off-by: Alejandro Lucero <alejandro.lucero@netronome.com>
>>>>>>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>>>>> ---
>>>>>>> lib/librte_eal/common/eal_common_memory.c | 33
>>>>>>> ++++++++++++++++++++++++++++++
>>>>>>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>>>>>>> lib/librte_eal/rte_eal_version.map | 1 +
>>>>>>> 3 files changed, 37 insertions(+)
>>>>>>>
>>>>>>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>>>>>>> b/lib/librte_eal/common/eal_common_memory.c
>>>>>>> index fc6c44d..f5efebe 100644
>>>>>>> --- a/lib/librte_eal/common/eal_common_memory.c
>>>>>>> +++ b/lib/librte_eal/common/eal_common_memory.c
>>>>>>> @@ -109,6 +109,39 @@
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> +/* check memseg iovas are within the required range based on dma
>>>>>>> mask
>>>>>>> */
>>>>>>> +int
>>>>>>> +rte_eal_check_dma_mask(uint8_t maskbits)
>>>>>>> +{
>>>>>>> +
>>>>>>> + const struct rte_mem_config *mcfg;
>>>>>>> + uint64_t mask;
>>>>>>> + int i;
>>>>>>> +
>>>>>>>
>>>>>>>
>>>>>>> I think we should add some sanity check to the input maskbits, i.e.
>>>>>> [64,0)
>>>>>> or [64, 32]? What would be a reasonable lower bound.
>>>>>>
>>>>>>
>>>>>> This is not a user's API, so any invocation will be reviewed, but I
>>>>>> guess
>>>>>>
>>>>> adding a sanity check here does not harm.
>>>>>
>>>>> Not sure about lower bound but upper should 64, although it does not
>>>>> make
>>>>> sense but it is safe. Lower bound is not so problematic.
>>>>>
>>>>>
>>>>>
>>>>> + /* create dma mask */
>>>>>>
>>>>>> + mask = ~((1ULL << maskbits) - 1);
>>>>>>> +
>>>>>>> + /* get pointer to global configuration */
>>>>>>> + mcfg = rte_eal_get_configuration()->mem_config;
>>>>>>> +
>>>>>>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>>>>>>> + if (mcfg->memseg[i].addr == NULL)
>>>>>>> + break;
>>>>>>>
>>>>>>>
>>>>>> Looking at some other code, it looks like NULL entries might exists.
>>>> So
>>>> should a continue; rather than a break; be used here?
>>>>
>>>>
>>>> I do not think so. memsegs are allocated sequentially, so first with
>>> addr
>>> as NULL implies no more memsegs.
>>>
>>
>> I was referring to the mem walk functions, rte_memseg_list_walk(). Maybe
>> some having more experience with this area can review/comment.
>>
>
> Pre-18.05, all memsegs are allocated continuously. Memseg lists and memseg
> list walk functions are 18.05+.
>
> Alejandro, perhaps it would be worth it to tag your patchset with
> "pre-18.05" to avoid similar confusion in the future?
>
>
Yes, that will help. I'm sending a new version shortly and I'll make it
clear.
> --
> Thanks,
> Anatoly
>
On 10 Jul 2018, at 13:43, Alejandro Lucero wrote:
> On Tue, Jul 10, 2018 at 12:33 PM, Burakov, Anatoly <
> anatoly.burakov@intel.com> wrote:
>
>> On 10-Jul-18 12:14 PM, Eelco Chaudron wrote:
>>
>>>
>>>
>>> On 10 Jul 2018, at 12:52, Alejandro Lucero wrote:
>>>
>>> On Tue, Jul 10, 2018 at 11:06 AM, Eelco Chaudron
>>> <echaudro@redhat.com>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> On 10 Jul 2018, at 11:34, Alejandro Lucero wrote:
>>>>>
>>>>> On Tue, Jul 10, 2018 at 9:56 AM, Eelco Chaudron
>>>>> <echaudro@redhat.com>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 4 Jul 2018, at 14:53, Alejandro Lucero wrote:
>>>>>>>
>>>>>>> A device can suffer addressing limitations. This functions
>>>>>>> checks
>>>>>>>
>>>>>>> memsegs have iovas within the supported range based on dma mask.
>>>>>>>>
>>>>>>>> PMD should use this during initialization if supported devices
>>>>>>>> suffer addressing limitations, returning an error if this
>>>>>>>> function
>>>>>>>> returns memsegs out of range.
>>>>>>>>
>>>>>>>> Another potential usage is for emulated IOMMU hardware with
>>>>>>>> addressing
>>>>>>>> limitations.
>>>>>>>>
>>>>>>>> Signed-off-by: Alejandro Lucero
>>>>>>>> <alejandro.lucero@netronome.com>
>>>>>>>> Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
>>>>>>>> ---
>>>>>>>> lib/librte_eal/common/eal_common_memory.c | 33
>>>>>>>> ++++++++++++++++++++++++++++++
>>>>>>>> lib/librte_eal/common/include/rte_memory.h | 3 +++
>>>>>>>> lib/librte_eal/rte_eal_version.map | 1 +
>>>>>>>> 3 files changed, 37 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/lib/librte_eal/common/eal_common_memory.c
>>>>>>>> b/lib/librte_eal/common/eal_common_memory.c
>>>>>>>> index fc6c44d..f5efebe 100644
>>>>>>>> --- a/lib/librte_eal/common/eal_common_memory.c
>>>>>>>> +++ b/lib/librte_eal/common/eal_common_memory.c
>>>>>>>> @@ -109,6 +109,39 @@
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> +/* check memseg iovas are within the required range based on
>>>>>>>> dma
>>>>>>>> mask
>>>>>>>> */
>>>>>>>> +int
>>>>>>>> +rte_eal_check_dma_mask(uint8_t maskbits)
>>>>>>>> +{
>>>>>>>> +
>>>>>>>> + const struct rte_mem_config *mcfg;
>>>>>>>> + uint64_t mask;
>>>>>>>> + int i;
>>>>>>>> +
>>>>>>>>
>>>>>>>>
>>>>>>>> I think we should add some sanity check to the input maskbits,
>>>>>>>> i.e.
>>>>>>> [64,0)
>>>>>>> or [64, 32]? What would be a reasonable lower bound.
>>>>>>>
>>>>>>>
>>>>>>> This is not a user's API, so any invocation will be reviewed,
>>>>>>> but I
>>>>>>> guess
>>>>>>>
>>>>>> adding a sanity check here does not harm.
>>>>>>
>>>>>> Not sure about lower bound but upper should 64, although it does
>>>>>> not
>>>>>> make
>>>>>> sense but it is safe. Lower bound is not so problematic.
>>>>>>
>>>>>>
>>>>>>
>>>>>> + /* create dma mask */
>>>>>>>
>>>>>>> + mask = ~((1ULL << maskbits) - 1);
>>>>>>>> +
>>>>>>>> + /* get pointer to global configuration */
>>>>>>>> + mcfg = rte_eal_get_configuration()->mem_config;
>>>>>>>> +
>>>>>>>> + for (i = 0; i < RTE_MAX_MEMSEG; i++) {
>>>>>>>> + if (mcfg->memseg[i].addr == NULL)
>>>>>>>> + break;
>>>>>>>>
>>>>>>>>
>>>>>>> Looking at some other code, it looks like NULL entries might
>>>>>>> exists.
>>>>> So
>>>>> should a continue; rather than a break; be used here?
>>>>>
>>>>>
>>>>> I do not think so. memsegs are allocated sequentially, so first
>>>>> with
>>>> addr
>>>> as NULL implies no more memsegs.
>>>>
>>>
>>> I was referring to the mem walk functions, rte_memseg_list_walk().
>>> Maybe
>>> some having more experience with this area can review/comment.
>>>
>>
>> Pre-18.05, all memsegs are allocated continuously. Memseg lists and
>> memseg
>> list walk functions are 18.05+.
>>
>> Alejandro, perhaps it would be worth it to tag your patchset with
>> "pre-18.05" to avoid similar confusion in the future?
>>
>>
> Yes, that will help. I'm sending a new version shortly and I'll make
> it
> clear.
Thanks, I’ll review the new version if it’s ready before the end of
tomorrow CET, as I will be on PTO.
@@ -109,6 +109,39 @@
}
}
+/* check memseg iovas are within the required range based on dma mask */
+int
+rte_eal_check_dma_mask(uint8_t maskbits)
+{
+
+ const struct rte_mem_config *mcfg;
+ uint64_t mask;
+ int i;
+
+ /* create dma mask */
+ mask = ~((1ULL << maskbits) - 1);
+
+ /* get pointer to global configuration */
+ mcfg = rte_eal_get_configuration()->mem_config;
+
+ for (i = 0; i < RTE_MAX_MEMSEG; i++) {
+ if (mcfg->memseg[i].addr == NULL)
+ break;
+
+ if (mcfg->memseg[i].iova & mask) {
+ RTE_LOG(INFO, EAL,
+ "memseg[%d] iova %"PRIx64" out of range:\n",
+ i, mcfg->memseg[i].iova);
+
+ RTE_LOG(INFO, EAL, "\tusing dma mask %"PRIx64"\n",
+ mask);
+ return -1;
+ }
+ }
+
+ return 0;
+}
+
/* return the number of memory channels */
unsigned rte_memory_get_nchannel(void)
{
@@ -209,6 +209,9 @@ struct rte_memseg {
*/
unsigned rte_memory_get_nrank(void);
+/* check memsegs iovas are within a range based on dma mask */
+int rte_eal_check_dma_mask(uint8_t maskbits);
+
/**
* Drivers based on uio will not load unless physical
* addresses are obtainable. It is only possible to get
@@ -184,6 +184,7 @@ DPDK_17.11 {
rte_eal_create_uio_dev;
rte_bus_get_iommu_class;
+ rte_eal_check_dma_mask;
rte_eal_has_pci;
rte_eal_iova_mode;
rte_eal_mbuf_default_mempool_ops;