From my reading of the code and docs this is what I understand: 1. Huge pages are needed by DPDK so that user pages are always present and pinned. 2. In systems with IOMMU and VFIO support, there should be no need for huge pages. VFIO_IOMMU_MAP_DMA on any aligned buffer should do the same pinning. --no-hugetlbfs today means that the system is also in --legacy-mem (which means no dynamic allocation, 1 fixed heap of 64MB MEMSIZE_IF_NO_HUGE). I think this is restrictive. On systems with IOMMU/VFIO support --no-hugetlbfs should also support dynamic(non legacy behavior) allocation.
Let's first clarify why huge pages are used. Memory pinning is possible with regular pages (subject to system limits). 1. Huge pages are needed: 1.1) to minimize TLB cache misses; 1.2) to increase chances of allocating large physically-contiguous blocks. 2. While 1.2) is irrelevant with IOMMU, 1.1) still affects performance. Using small pages would also consume more IOMMU entries, depleting them and degrading performance even worse (IOMMU has to translate VA to PA despite the buffer is VA-contiguous). The --no-huge mode is intended to run (unit)tests with minimal requirements from the system, not even privileged access, including IOMMU access. That being said, enabling dynamic --no-huge should be possible and even desirable in the course of gradually moving to dynamic mode entirely. What is your use case that you want to alleviate this restriction?
I agree that huge pages offer performance benefits. The reason why I filed this bug is that we are trying to figure out "guaranteed" huge page support in our containers to the extent we need them. That is the ability to reserve many huge pages in our container is best effort. And therefore I have been looking into falling back to --no-huge mode (accepting the degraded performance here) in a production scenario.