[dpdk-stable] [dpdk-dev] [PATCH] memory: do not use base-virtaddr in secondary processes

[Burakov, Anatoly]

On 19-Jun-18 11:23 AM, Alejandro Lucero wrote:
> 
> 
> On Tue, Jun 19, 2018 at 10:24 AM, Burakov, Anatoly 
> <anatoly.burakov at intel.com <mailto:anatoly.burakov at intel.com>> wrote:
> 
>     On 18-Jun-18 9:12 PM, Stojaczyk, DariuszX wrote:
> 
> 
> 
>             -----Original Message-----
>             From: Alejandro Lucero
>             [mailto:alejandro.lucero at netronome.com
>             <mailto:alejandro.lucero at netronome.com>]
>             Sent: Monday, June 18, 2018 9:33 PM
> 
>             On Mon, Jun 18, 2018 at 8:03 PM, Stojaczyk, DariuszX
>             <dariuszx.stojaczyk at intel.com
>             <mailto:dariuszx.stojaczyk at intel.com>
>             <mailto:dariuszx.stojaczyk at intel.com
>             <mailto:dariuszx.stojaczyk at intel.com>> >
>             wrote:
> 
>                      Can you point me out to an NFP guide or some code
>             that describes
>             this in more detail?
> 
> 
> 
>             As I said, I'm working on a RFC. I will send something
>             shortly. But I could give
>             you an advance: the hugepages needs to be mapped below
>             certain virtual
>             address, 1TB, and I'm afraid that includes the primary and
>             also the
>             secondary processes. At least if any process can send or
>             receive packets
>             to/from a NFP.
> 
> 
> 
>         Thanks, I'm pretty sure we're safe, then.
> 
> 
>                      If we're talking about base-virtaddr for hugepages,
>             then that's always
>             inherited from the primary process, regardless of what
>             base-virtaddr is
>             supplied to the secondary.
> 
> 
> 
> 
>             But, is not your patch avoiding to use that base-virtaddr
>             for secondary
>             processes?
> 
> 
>         I see now that the patch name is slightly misleading. Maybe I
>         shouldn’t pick such a catchy title. Let me clarify: As of DPDK
>         18.05, --base-virtaddr param for secondary process applications
>         only affects that shadow memseg metadata that's not useful for
>         anyone, but can still do a lot of harm. Hugepage memory in
>         secondary processes is always mapped to the same addresses the
>         primary process uses.
> 
>         D.
> 
> 
>     Hi Alejandro,
> 
>     To solve this problem, one possible approach would be to have
>     maximum VA address, and allocate memory downwards, rather than
>     upwards. Is that by any chance approximate contents of your RFC? :)
> 
> 
> Hi Anatoly,
> 
> There's a lot of space below 1TB, but this specific upper limit is just 
> in the NFP case. My RFC will propose a generic solution assuming there 
> could be other devices in the future, and not just NIC devices, which 
> could have also some sort of limitation.

OK, looking forward to it.

> The problem is, those devices 
> limitations can not be known at memory initialization time, so some sort 
> of check is required afterwards, once the devices have been proved and 
> initialised.
> 

Presumably, device driver would be in the best position to know things 
like that, no? Maybe a new device flag and an API to query such things 
from all devices?

-- 
Thanks,
Anatoly