[v6,04/21] mem: do not check for invalid socket ID

Message ID ecf14354d0a7c1c4f2bb03c66cd739e849fcfce9.1538044725.git.anatoly.burakov@intel.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series Support externally allocated memory in DPDK |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Anatoly Burakov Sept. 27, 2018, 10:41 a.m. UTC
  We will be assigning "invalid" socket ID's to external heap, and
malloc will now be able to verify if a supplied socket ID is in
fact a valid one, rendering parameter checks for sockets
obsolete.

This changes the semantics of what we understand by "socket ID",
so document the change in the release notes.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
 doc/guides/rel_notes/release_18_11.rst     | 7 +++++++
 lib/librte_eal/common/eal_common_memzone.c | 8 +++++---
 lib/librte_eal/common/malloc_heap.c        | 2 +-
 lib/librte_eal/common/rte_malloc.c         | 4 ----
 4 files changed, 13 insertions(+), 8 deletions(-)
  

Comments

Alejandro Lucero Sept. 27, 2018, 1:14 p.m. UTC | #1
On Thu, Sep 27, 2018 at 11:41 AM Anatoly Burakov <anatoly.burakov@intel.com>
wrote:

> We will be assigning "invalid" socket ID's to external heap, and
> malloc will now be able to verify if a supplied socket ID is in
> fact a valid one, rendering parameter checks for sockets
> obsolete.
>
> This changes the semantics of what we understand by "socket ID",
> so document the change in the release notes.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>  doc/guides/rel_notes/release_18_11.rst     | 7 +++++++
>  lib/librte_eal/common/eal_common_memzone.c | 8 +++++---
>  lib/librte_eal/common/malloc_heap.c        | 2 +-
>  lib/librte_eal/common/rte_malloc.c         | 4 ----
>  4 files changed, 13 insertions(+), 8 deletions(-)
>
> diff --git a/doc/guides/rel_notes/release_18_11.rst
> b/doc/guides/rel_notes/release_18_11.rst
> index 5fc71e208..6ee236302 100644
> --- a/doc/guides/rel_notes/release_18_11.rst
> +++ b/doc/guides/rel_notes/release_18_11.rst
> @@ -98,6 +98,13 @@ API Changes
>      users of memseg-walk-related functions, as they will now have to skip
>      externally allocated segments in most cases if the intent is to only
> iterate
>      over internal DPDK memory.
> +  - ``socket_id`` parameter across the entire DPDK has gained additional
> +    meaning, as some socket ID's will now be representing externally
> allocated
> +    memory. No changes will be required for existing code as backwards
> +    compatibility will be kept, and those who do not use this feature
> will not
> +    see these extra socket ID's. Any new API's must not check socket ID
> +    parameters themselves, and must instead leave it to the memory
> subsystem to
> +    decide whether socket ID is a valid one.
>
>  ABI Changes
>  -----------
> diff --git a/lib/librte_eal/common/eal_common_memzone.c
> b/lib/librte_eal/common/eal_common_memzone.c
> index 7300fe05d..b7081afbf 100644
> --- a/lib/librte_eal/common/eal_common_memzone.c
> +++ b/lib/librte_eal/common/eal_common_memzone.c
> @@ -120,13 +120,15 @@ memzone_reserve_aligned_thread_unsafe(const char
> *name, size_t len,
>                 return NULL;
>         }
>
> -       if ((socket_id != SOCKET_ID_ANY) &&
> -           (socket_id >= RTE_MAX_NUMA_NODES || socket_id < 0)) {
> +       if ((socket_id != SOCKET_ID_ANY) && socket_id < 0) {
>

Should not it be better to use RTE_MAX_HEAP instead of removing the check?



>                 rte_errno = EINVAL;
>                 return NULL;
>         }
>
> -       if (!rte_eal_has_hugepages())
> +       /* only set socket to SOCKET_ID_ANY if we aren't allocating for an
> +        * external heap.
> +        */
> +       if (!rte_eal_has_hugepages() && socket_id < RTE_MAX_NUMA_NODES)
>                 socket_id = SOCKET_ID_ANY;
>
>         contig = (flags & RTE_MEMZONE_IOVA_CONTIG) != 0;
> diff --git a/lib/librte_eal/common/malloc_heap.c
> b/lib/librte_eal/common/malloc_heap.c
> index 1d1e35708..73e478076 100644
> --- a/lib/librte_eal/common/malloc_heap.c
> +++ b/lib/librte_eal/common/malloc_heap.c
> @@ -647,7 +647,7 @@ malloc_heap_alloc(const char *type, size_t size, int
> socket_arg,
>         if (size == 0 || (align && !rte_is_power_of_2(align)))
>                 return NULL;
>
> -       if (!rte_eal_has_hugepages())
> +       if (!rte_eal_has_hugepages() && socket_arg < RTE_MAX_NUMA_NODES)
>                 socket_arg = SOCKET_ID_ANY;
>
>         if (socket_arg == SOCKET_ID_ANY)
> diff --git a/lib/librte_eal/common/rte_malloc.c
> b/lib/librte_eal/common/rte_malloc.c
> index 73d6df31d..9ba1472c3 100644
> --- a/lib/librte_eal/common/rte_malloc.c
> +++ b/lib/librte_eal/common/rte_malloc.c
> @@ -47,10 +47,6 @@ rte_malloc_socket(const char *type, size_t size,
> unsigned int align,
>         if (!rte_eal_has_hugepages())
>                 socket_arg = SOCKET_ID_ANY;
>
> -       /* Check socket parameter */
> -       if (socket_arg >= RTE_MAX_NUMA_NODES)
> -               return NULL;
> -
>

Sane than before. Better to keep the sanity check using RTE_MAX_HEAPS.


>         return malloc_heap_alloc(type, size, socket_arg, 0,
>                         align == 0 ? 1 : align, 0, false);
>  }
> --
> 2.17.1
>
  
Anatoly Burakov Sept. 27, 2018, 1:21 p.m. UTC | #2
On 27-Sep-18 2:14 PM, Alejandro Lucero wrote:
> On Thu, Sep 27, 2018 at 11:41 AM Anatoly Burakov <anatoly.burakov@intel.com>
> wrote:
> 
>> We will be assigning "invalid" socket ID's to external heap, and
>> malloc will now be able to verify if a supplied socket ID is in
>> fact a valid one, rendering parameter checks for sockets
>> obsolete.
>>
>> This changes the semantics of what we understand by "socket ID",
>> so document the change in the release notes.
>>
>> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>> ---
>>   doc/guides/rel_notes/release_18_11.rst     | 7 +++++++
>>   lib/librte_eal/common/eal_common_memzone.c | 8 +++++---
>>   lib/librte_eal/common/malloc_heap.c        | 2 +-
>>   lib/librte_eal/common/rte_malloc.c         | 4 ----
>>   4 files changed, 13 insertions(+), 8 deletions(-)
>>
>> diff --git a/doc/guides/rel_notes/release_18_11.rst
>> b/doc/guides/rel_notes/release_18_11.rst
>> index 5fc71e208..6ee236302 100644
>> --- a/doc/guides/rel_notes/release_18_11.rst
>> +++ b/doc/guides/rel_notes/release_18_11.rst
>> @@ -98,6 +98,13 @@ API Changes
>>       users of memseg-walk-related functions, as they will now have to skip
>>       externally allocated segments in most cases if the intent is to only
>> iterate
>>       over internal DPDK memory.
>> +  - ``socket_id`` parameter across the entire DPDK has gained additional
>> +    meaning, as some socket ID's will now be representing externally
>> allocated
>> +    memory. No changes will be required for existing code as backwards
>> +    compatibility will be kept, and those who do not use this feature
>> will not
>> +    see these extra socket ID's. Any new API's must not check socket ID
>> +    parameters themselves, and must instead leave it to the memory
>> subsystem to
>> +    decide whether socket ID is a valid one.
>>
>>   ABI Changes
>>   -----------
>> diff --git a/lib/librte_eal/common/eal_common_memzone.c
>> b/lib/librte_eal/common/eal_common_memzone.c
>> index 7300fe05d..b7081afbf 100644
>> --- a/lib/librte_eal/common/eal_common_memzone.c
>> +++ b/lib/librte_eal/common/eal_common_memzone.c
>> @@ -120,13 +120,15 @@ memzone_reserve_aligned_thread_unsafe(const char
>> *name, size_t len,
>>                  return NULL;
>>          }
>>
>> -       if ((socket_id != SOCKET_ID_ANY) &&
>> -           (socket_id >= RTE_MAX_NUMA_NODES || socket_id < 0)) {
>> +       if ((socket_id != SOCKET_ID_ANY) && socket_id < 0) {
>>
> 
> Should not it be better to use RTE_MAX_HEAP instead of removing the check?

First of all, maximum number of heaps should not concern the rest of the 
code - this is purely internal detail of rte_malloc.

More importantly, socket ID is completely independent from number of 
heaps. Socket ID is incremented each time a new heap is created, and 
they are not reused. If you create and destroy a heap 100 times - you'll 
get 100 different socket ID's, even though max number of heaps is less 
than that.

> 
> 
> 
>>                  rte_errno = EINVAL;
>>                  return NULL;
>>          }
>>
>> -       if (!rte_eal_has_hugepages())
>> +       /* only set socket to SOCKET_ID_ANY if we aren't allocating for an
>> +        * external heap.
>> +        */
>> +       if (!rte_eal_has_hugepages() && socket_id < RTE_MAX_NUMA_NODES)
>>                  socket_id = SOCKET_ID_ANY;
>>
>>          contig = (flags & RTE_MEMZONE_IOVA_CONTIG) != 0;
>> diff --git a/lib/librte_eal/common/malloc_heap.c
>> b/lib/librte_eal/common/malloc_heap.c
>> index 1d1e35708..73e478076 100644
>> --- a/lib/librte_eal/common/malloc_heap.c
>> +++ b/lib/librte_eal/common/malloc_heap.c
>> @@ -647,7 +647,7 @@ malloc_heap_alloc(const char *type, size_t size, int
>> socket_arg,
>>          if (size == 0 || (align && !rte_is_power_of_2(align)))
>>                  return NULL;
>>
>> -       if (!rte_eal_has_hugepages())
>> +       if (!rte_eal_has_hugepages() && socket_arg < RTE_MAX_NUMA_NODES)
>>                  socket_arg = SOCKET_ID_ANY;
>>
>>          if (socket_arg == SOCKET_ID_ANY)
>> diff --git a/lib/librte_eal/common/rte_malloc.c
>> b/lib/librte_eal/common/rte_malloc.c
>> index 73d6df31d..9ba1472c3 100644
>> --- a/lib/librte_eal/common/rte_malloc.c
>> +++ b/lib/librte_eal/common/rte_malloc.c
>> @@ -47,10 +47,6 @@ rte_malloc_socket(const char *type, size_t size,
>> unsigned int align,
>>          if (!rte_eal_has_hugepages())
>>                  socket_arg = SOCKET_ID_ANY;
>>
>> -       /* Check socket parameter */
>> -       if (socket_arg >= RTE_MAX_NUMA_NODES)
>> -               return NULL;
>> -
>>
> 
> Sane than before. Better to keep the sanity check using RTE_MAX_HEAPS.

same as above :)
  
Alejandro Lucero Sept. 27, 2018, 1:42 p.m. UTC | #3
On Thu, Sep 27, 2018 at 2:22 PM Burakov, Anatoly <anatoly.burakov@intel.com>
wrote:

> On 27-Sep-18 2:14 PM, Alejandro Lucero wrote:
> > On Thu, Sep 27, 2018 at 11:41 AM Anatoly Burakov <
> anatoly.burakov@intel.com>
> > wrote:
> >
> >> We will be assigning "invalid" socket ID's to external heap, and
> >> malloc will now be able to verify if a supplied socket ID is in
> >> fact a valid one, rendering parameter checks for sockets
> >> obsolete.
> >>
> >> This changes the semantics of what we understand by "socket ID",
> >> so document the change in the release notes.
> >>
> >> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> >> ---
> >>   doc/guides/rel_notes/release_18_11.rst     | 7 +++++++
> >>   lib/librte_eal/common/eal_common_memzone.c | 8 +++++---
> >>   lib/librte_eal/common/malloc_heap.c        | 2 +-
> >>   lib/librte_eal/common/rte_malloc.c         | 4 ----
> >>   4 files changed, 13 insertions(+), 8 deletions(-)
> >>
> >> diff --git a/doc/guides/rel_notes/release_18_11.rst
> >> b/doc/guides/rel_notes/release_18_11.rst
> >> index 5fc71e208..6ee236302 100644
> >> --- a/doc/guides/rel_notes/release_18_11.rst
> >> +++ b/doc/guides/rel_notes/release_18_11.rst
> >> @@ -98,6 +98,13 @@ API Changes
> >>       users of memseg-walk-related functions, as they will now have to
> skip
> >>       externally allocated segments in most cases if the intent is to
> only
> >> iterate
> >>       over internal DPDK memory.
> >> +  - ``socket_id`` parameter across the entire DPDK has gained
> additional
> >> +    meaning, as some socket ID's will now be representing externally
> >> allocated
> >> +    memory. No changes will be required for existing code as backwards
> >> +    compatibility will be kept, and those who do not use this feature
> >> will not
> >> +    see these extra socket ID's. Any new API's must not check socket ID
> >> +    parameters themselves, and must instead leave it to the memory
> >> subsystem to
> >> +    decide whether socket ID is a valid one.
> >>
> >>   ABI Changes
> >>   -----------
> >> diff --git a/lib/librte_eal/common/eal_common_memzone.c
> >> b/lib/librte_eal/common/eal_common_memzone.c
> >> index 7300fe05d..b7081afbf 100644
> >> --- a/lib/librte_eal/common/eal_common_memzone.c
> >> +++ b/lib/librte_eal/common/eal_common_memzone.c
> >> @@ -120,13 +120,15 @@ memzone_reserve_aligned_thread_unsafe(const char
> >> *name, size_t len,
> >>                  return NULL;
> >>          }
> >>
> >> -       if ((socket_id != SOCKET_ID_ANY) &&
> >> -           (socket_id >= RTE_MAX_NUMA_NODES || socket_id < 0)) {
> >> +       if ((socket_id != SOCKET_ID_ANY) && socket_id < 0) {
> >>
> >
> > Should not it be better to use RTE_MAX_HEAP instead of removing the
> check?
>
> First of all, maximum number of heaps should not concern the rest of the
> code - this is purely internal detail of rte_malloc.
>
>
In a previous patch you say that:

"Switch over all parts of EAL to use heap ID instead of NUMA node
ID to identify heaps. Heap ID for DPDK-internal heaps is NUMA
node's index within the detected NUMA node list. Heap ID for
external heaps will be order of their creation."

If I understand this right, heaps linked to physical sockets get a heap ID,
and then external heaps will get IDs starting from the higher socket/heap
ID + 1.
So, assuming RTE_MAX_HEAPS is really the maximum number of allowed heaps
(which does not seem so reading your next paragraph), it would be a good
sanity check to use RTE_MAX_HEAPS for the socket id.

More importantly, socket ID is completely independent from number of
> heaps. Socket ID is incremented each time a new heap is created, and
> they are not reused. If you create and destroy a heap 100 times - you'll
> get 100 different socket ID's, even though max number of heaps is less
> than that.
>
>
I do not understand this. It is true there is no check regarding
RTE_MAX_HEAPS when creating new heaps, then nor sure what the limit refers
to. And then there is code like dumping heaps info or getting info from the
heap based on socket id that will not work.


> >
> >
> >
> >>                  rte_errno = EINVAL;
> >>                  return NULL;
> >>          }
> >>
> >> -       if (!rte_eal_has_hugepages())
> >> +       /* only set socket to SOCKET_ID_ANY if we aren't allocating for
> an
> >> +        * external heap.
> >> +        */
> >> +       if (!rte_eal_has_hugepages() && socket_id < RTE_MAX_NUMA_NODES)
> >>                  socket_id = SOCKET_ID_ANY;
> >>
> >>          contig = (flags & RTE_MEMZONE_IOVA_CONTIG) != 0;
> >> diff --git a/lib/librte_eal/common/malloc_heap.c
> >> b/lib/librte_eal/common/malloc_heap.c
> >> index 1d1e35708..73e478076 100644
> >> --- a/lib/librte_eal/common/malloc_heap.c
> >> +++ b/lib/librte_eal/common/malloc_heap.c
> >> @@ -647,7 +647,7 @@ malloc_heap_alloc(const char *type, size_t size, int
> >> socket_arg,
> >>          if (size == 0 || (align && !rte_is_power_of_2(align)))
> >>                  return NULL;
> >>
> >> -       if (!rte_eal_has_hugepages())
> >> +       if (!rte_eal_has_hugepages() && socket_arg < RTE_MAX_NUMA_NODES)
> >>                  socket_arg = SOCKET_ID_ANY;
> >>
> >>          if (socket_arg == SOCKET_ID_ANY)
> >> diff --git a/lib/librte_eal/common/rte_malloc.c
> >> b/lib/librte_eal/common/rte_malloc.c
> >> index 73d6df31d..9ba1472c3 100644
> >> --- a/lib/librte_eal/common/rte_malloc.c
> >> +++ b/lib/librte_eal/common/rte_malloc.c
> >> @@ -47,10 +47,6 @@ rte_malloc_socket(const char *type, size_t size,
> >> unsigned int align,
> >>          if (!rte_eal_has_hugepages())
> >>                  socket_arg = SOCKET_ID_ANY;
> >>
> >> -       /* Check socket parameter */
> >> -       if (socket_arg >= RTE_MAX_NUMA_NODES)
> >> -               return NULL;
> >> -
> >>
> >
> > Sane than before. Better to keep the sanity check using RTE_MAX_HEAPS.
>
> same as above :)
>
>
> --
> Thanks,
> Anatoly
>
  
Anatoly Burakov Sept. 27, 2018, 2:04 p.m. UTC | #4
On 27-Sep-18 2:42 PM, Alejandro Lucero wrote:
> 
> 
> On Thu, Sep 27, 2018 at 2:22 PM Burakov, Anatoly 
> <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>> wrote:
> 
>     On 27-Sep-18 2:14 PM, Alejandro Lucero wrote:
>      > On Thu, Sep 27, 2018 at 11:41 AM Anatoly Burakov
>     <anatoly.burakov@intel.com <mailto:anatoly.burakov@intel.com>>
>      > wrote:
>      >
>      >> We will be assigning "invalid" socket ID's to external heap, and
>      >> malloc will now be able to verify if a supplied socket ID is in
>      >> fact a valid one, rendering parameter checks for sockets
>      >> obsolete.
>      >>
>      >> This changes the semantics of what we understand by "socket ID",
>      >> so document the change in the release notes.
>      >>
>      >> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com
>     <mailto:anatoly.burakov@intel.com>>
>      >> ---
>      >>   doc/guides/rel_notes/release_18_11.rst     | 7 +++++++
>      >>   lib/librte_eal/common/eal_common_memzone.c | 8 +++++---
>      >>   lib/librte_eal/common/malloc_heap.c        | 2 +-
>      >>   lib/librte_eal/common/rte_malloc.c         | 4 ----
>      >>   4 files changed, 13 insertions(+), 8 deletions(-)
>      >>
>      >> diff --git a/doc/guides/rel_notes/release_18_11.rst
>      >> b/doc/guides/rel_notes/release_18_11.rst
>      >> index 5fc71e208..6ee236302 100644
>      >> --- a/doc/guides/rel_notes/release_18_11.rst
>      >> +++ b/doc/guides/rel_notes/release_18_11.rst
>      >> @@ -98,6 +98,13 @@ API Changes
>      >>       users of memseg-walk-related functions, as they will now
>     have to skip
>      >>       externally allocated segments in most cases if the intent
>     is to only
>      >> iterate
>      >>       over internal DPDK memory.
>      >> +  - ``socket_id`` parameter across the entire DPDK has gained
>     additional
>      >> +    meaning, as some socket ID's will now be representing
>     externally
>      >> allocated
>      >> +    memory. No changes will be required for existing code as
>     backwards
>      >> +    compatibility will be kept, and those who do not use this
>     feature
>      >> will not
>      >> +    see these extra socket ID's. Any new API's must not check
>     socket ID
>      >> +    parameters themselves, and must instead leave it to the memory
>      >> subsystem to
>      >> +    decide whether socket ID is a valid one.
>      >>
>      >>   ABI Changes
>      >>   -----------
>      >> diff --git a/lib/librte_eal/common/eal_common_memzone.c
>      >> b/lib/librte_eal/common/eal_common_memzone.c
>      >> index 7300fe05d..b7081afbf 100644
>      >> --- a/lib/librte_eal/common/eal_common_memzone.c
>      >> +++ b/lib/librte_eal/common/eal_common_memzone.c
>      >> @@ -120,13 +120,15 @@
>     memzone_reserve_aligned_thread_unsafe(const char
>      >> *name, size_t len,
>      >>                  return NULL;
>      >>          }
>      >>
>      >> -       if ((socket_id != SOCKET_ID_ANY) &&
>      >> -           (socket_id >= RTE_MAX_NUMA_NODES || socket_id < 0)) {
>      >> +       if ((socket_id != SOCKET_ID_ANY) && socket_id < 0) {
>      >>
>      >
>      > Should not it be better to use RTE_MAX_HEAP instead of removing
>     the check?
> 
>     First of all, maximum number of heaps should not concern the rest of
>     the
>     code - this is purely internal detail of rte_malloc.
> 
> 
> In a previous patch you say that:
> 
> "Switch over all parts of EAL to use heap ID instead of NUMA node
> ID to identify heaps. Heap ID for DPDK-internal heaps is NUMA
> node's index within the detected NUMA node list. Heap ID for
> external heaps will be order of their creation."
> 
> If I understand this right, heaps linked to physical sockets get a heap 
> ID, and then external heaps will get IDs starting from the higher 
> socket/heap ID + 1.

Yes and no.

Socket ID is an externally visible identification of "where to allocate 
from" (a heap). Heap ID is used internally. Normally, there is a 1:1 
correspondence of NUMA node to heap ID, but there may be cases where 
e.g. only NUMA nodes 0 and 7 are detected, so you'll have socket 0 and 7 
as valid socket ID's. However, these socket ID's will be internally 
resolved into heap ID's 0 and 1, not 0 and 7.

So, in *most* cases, socket ID for an internal heap is equivalent to its 
heap ID, but it is by accident. Heap ID is an internal identifier used 
by the malloc heap, and it is not visible externally - it is only known 
to malloc itself. Even memzone knows nothing about heap ID's - only 
socket ID's.

> So, assuming RTE_MAX_HEAPS is really the maximum number of allowed heaps 
> (which does not seem so reading your next paragraph), it would be a good 
> sanity check to use RTE_MAX_HEAPS for the socket id.
> 
>     More importantly, socket ID is completely independent from number of
>     heaps. Socket ID is incremented each time a new heap is created, and
>     they are not reused. If you create and destroy a heap 100 times -
>     you'll
>     get 100 different socket ID's, even though max number of heaps is less
>     than that.
> 
> 
> I do not understand this. It is true there is no check regarding 
> RTE_MAX_HEAPS when creating new heaps,

There is one :) RTE_MAX_HEAPS is length of malloc heaps array (shared in 
memory). If we cannot find a vacant spot in heaps array, the heap will 
not be created.

However, *socket ID* is indeed limited only to INT_MAX. Socket ID is not 
heap ID - socket ID is an externally visible identifier. Multiple socket 
ID's can resolve to the same heap ID.

For example, if you create and destroy a heap 5 times one after the 
other, you'll get 5 different socket ID's, but all of them would have 
pointed to the same heap ID (but not at the same time).

So, semantically speaking, heap ID isn't really "an ID" as such, it's an 
index into heap array. Unlike socket ID, it has no meaning.

> then nor sure what the limit 
> refers to. And then there is code like dumping heaps info or getting 
> info from the heap based on socket id that will not work.

It is probably unclear because the ordering of this patchset is not 
ideal (and i'm not sure how to make it any better).

The code for dumping or getting heap info's accepts socket ID, but it 
translates it into heap ID, because that's what malloc uses internally 
to differentiate between the heaps. Heap ID is there to break dependency 
between NUMA node ID and position in the malloc heap array.
  

Patch

diff --git a/doc/guides/rel_notes/release_18_11.rst b/doc/guides/rel_notes/release_18_11.rst
index 5fc71e208..6ee236302 100644
--- a/doc/guides/rel_notes/release_18_11.rst
+++ b/doc/guides/rel_notes/release_18_11.rst
@@ -98,6 +98,13 @@  API Changes
     users of memseg-walk-related functions, as they will now have to skip
     externally allocated segments in most cases if the intent is to only iterate
     over internal DPDK memory.
+  - ``socket_id`` parameter across the entire DPDK has gained additional
+    meaning, as some socket ID's will now be representing externally allocated
+    memory. No changes will be required for existing code as backwards
+    compatibility will be kept, and those who do not use this feature will not
+    see these extra socket ID's. Any new API's must not check socket ID
+    parameters themselves, and must instead leave it to the memory subsystem to
+    decide whether socket ID is a valid one.
 
 ABI Changes
 -----------
diff --git a/lib/librte_eal/common/eal_common_memzone.c b/lib/librte_eal/common/eal_common_memzone.c
index 7300fe05d..b7081afbf 100644
--- a/lib/librte_eal/common/eal_common_memzone.c
+++ b/lib/librte_eal/common/eal_common_memzone.c
@@ -120,13 +120,15 @@  memzone_reserve_aligned_thread_unsafe(const char *name, size_t len,
 		return NULL;
 	}
 
-	if ((socket_id != SOCKET_ID_ANY) &&
-	    (socket_id >= RTE_MAX_NUMA_NODES || socket_id < 0)) {
+	if ((socket_id != SOCKET_ID_ANY) && socket_id < 0) {
 		rte_errno = EINVAL;
 		return NULL;
 	}
 
-	if (!rte_eal_has_hugepages())
+	/* only set socket to SOCKET_ID_ANY if we aren't allocating for an
+	 * external heap.
+	 */
+	if (!rte_eal_has_hugepages() && socket_id < RTE_MAX_NUMA_NODES)
 		socket_id = SOCKET_ID_ANY;
 
 	contig = (flags & RTE_MEMZONE_IOVA_CONTIG) != 0;
diff --git a/lib/librte_eal/common/malloc_heap.c b/lib/librte_eal/common/malloc_heap.c
index 1d1e35708..73e478076 100644
--- a/lib/librte_eal/common/malloc_heap.c
+++ b/lib/librte_eal/common/malloc_heap.c
@@ -647,7 +647,7 @@  malloc_heap_alloc(const char *type, size_t size, int socket_arg,
 	if (size == 0 || (align && !rte_is_power_of_2(align)))
 		return NULL;
 
-	if (!rte_eal_has_hugepages())
+	if (!rte_eal_has_hugepages() && socket_arg < RTE_MAX_NUMA_NODES)
 		socket_arg = SOCKET_ID_ANY;
 
 	if (socket_arg == SOCKET_ID_ANY)
diff --git a/lib/librte_eal/common/rte_malloc.c b/lib/librte_eal/common/rte_malloc.c
index 73d6df31d..9ba1472c3 100644
--- a/lib/librte_eal/common/rte_malloc.c
+++ b/lib/librte_eal/common/rte_malloc.c
@@ -47,10 +47,6 @@  rte_malloc_socket(const char *type, size_t size, unsigned int align,
 	if (!rte_eal_has_hugepages())
 		socket_arg = SOCKET_ID_ANY;
 
-	/* Check socket parameter */
-	if (socket_arg >= RTE_MAX_NUMA_NODES)
-		return NULL;
-
 	return malloc_heap_alloc(type, size, socket_arg, 0,
 			align == 0 ? 1 : align, 0, false);
 }