[dpdk-dev,v3,3/3] mem: improve autodetection of hugepage counts on 32-bit
Checks
Commit Message
For non-legacy mode, we are preallocating space for hugepages, so
we know in advance which pages we will be able to allocate, and
which we won't. However, the init procedure was using hugepage
counts gathered from sysfs and paid no attention to hugepage
sizes that were actually available for reservation, and failed
on attempts to reserve unavailable pages.
Fix this by limiting total page counts by number of pages
actually preallocated.
Also, VA preallocate procedure only looks at mountpoints that are
available, and expects pages to exist if a mountpoint exists. That
might not necessarily be the case, so also check if there are
hugepages available for a particular page size on a particular
NUMA node.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
---
Notes:
v3:
- Added a check to see if pages are actually available before
reserving VA space for them. Only applies to 32-bit non-legacy.
lib/librte_eal/common/eal_common_memory.c | 4 ++++
lib/librte_eal/linuxapp/eal/eal_memory.c | 31 +++++++++++++++++++++++++++++++
2 files changed, 35 insertions(+)
Comments
> -----Original Message-----
> From: Burakov, Anatoly
> Sent: Tuesday, April 24, 2018 11:19 AM
> To: dev@dpdk.org
> Cc: Pattan, Reshma <reshma.pattan@intel.com>
> Subject: [PATCH v3 3/3] mem: improve autodetection of hugepage counts on
> 32-bit
>
> For non-legacy mode, we are preallocating space for hugepages, so we know
> in advance which pages we will be able to allocate, and which we won't.
> However, the init procedure was using hugepage counts gathered from sysfs
> and paid no attention to hugepage sizes that were actually available for
> reservation, and failed on attempts to reserve unavailable pages.
>
> Fix this by limiting total page counts by number of pages actually
> preallocated.
>
> Also, VA preallocate procedure only looks at mountpoints that are available,
> and expects pages to exist if a mountpoint exists. That might not necessarily
> be the case, so also check if there are hugepages available for a particular
> page size on a particular NUMA node.
>
> Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
> ---
>
Tested-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
> > For non-legacy mode, we are preallocating space for hugepages, so we know
> > in advance which pages we will be able to allocate, and which we won't.
> > However, the init procedure was using hugepage counts gathered from sysfs
> > and paid no attention to hugepage sizes that were actually available for
> > reservation, and failed on attempts to reserve unavailable pages.
> >
> > Fix this by limiting total page counts by number of pages actually
> > preallocated.
> >
> > Also, VA preallocate procedure only looks at mountpoints that are available,
> > and expects pages to exist if a mountpoint exists. That might not necessarily
> > be the case, so also check if there are hugepages available for a particular
> > page size on a particular NUMA node.
> >
> > Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
>
> Tested-by: Jananee Parthasarathy <jananeex.m.parthasarathy@intel.com>
Series applied, thanks
@@ -329,6 +329,10 @@ memseg_primary_init_32(void)
hpi = &internal_config.hugepage_info[hpi_idx];
hugepage_sz = hpi->hugepage_sz;
+ /* check if pages are actually available */
+ if (hpi->num_pages[socket_id] == 0)
+ continue;
+
max_segs = RTE_MAX_MEMSEG_PER_TYPE;
max_pagesz_mem = max_socket_mem - cur_socket_mem;
@@ -1603,6 +1603,18 @@ eal_legacy_hugepage_init(void)
return -1;
}
+static int __rte_unused
+hugepage_count_walk(const struct rte_memseg_list *msl, void *arg)
+{
+ struct hugepage_info *hpi = arg;
+
+ if (msl->page_sz != hpi->hugepage_sz)
+ return 0;
+
+ hpi->num_pages[msl->socket_id] += msl->memseg_arr.len;
+ return 0;
+}
+
static int
eal_hugepage_init(void)
{
@@ -1617,10 +1629,29 @@ eal_hugepage_init(void)
for (hp_sz_idx = 0;
hp_sz_idx < (int) internal_config.num_hugepage_sizes;
hp_sz_idx++) {
+#ifndef RTE_ARCH_64
+ struct hugepage_info dummy;
+ unsigned int i;
+#endif
/* also initialize used_hp hugepage sizes in used_hp */
struct hugepage_info *hpi;
hpi = &internal_config.hugepage_info[hp_sz_idx];
used_hp[hp_sz_idx].hugepage_sz = hpi->hugepage_sz;
+
+#ifndef RTE_ARCH_64
+ /* for 32-bit, limit number of pages on socket to whatever we've
+ * preallocated, as we cannot allocate more.
+ */
+ memset(&dummy, 0, sizeof(dummy));
+ dummy.hugepage_sz = hpi->hugepage_sz;
+ if (rte_memseg_list_walk(hugepage_count_walk, &dummy) < 0)
+ return -1;
+
+ for (i = 0; i < RTE_DIM(dummy.num_pages); i++) {
+ hpi->num_pages[i] = RTE_MIN(hpi->num_pages[i],
+ dummy.num_pages[i]);
+ }
+#endif
}
/* make a copy of socket_mem, needed for balanced allocation. */