[SPDK] --no-huge on IOMMU/VFIO platform

niketkandya at google.com Fri, 03 Dec 2021 02:12:15 +0000
Newsgroups dev.linux.lists.spdk
Message-ID <[email protected]>
From my reading of the code and docs this is what I understand:
1. Huge pages are needed by SPDK/DPDK so that user pages are always present and pinned.
2. In systems with IOMMU and VFIO support, there should be no need for huge pages. spdk_mem_register on an externally allocated buffer should do the same pinning.
3. This is also what happens to DPDK's internal heaps in the memory callbacks (memory_iter_cb).

When I tried to use --no-huge with SPDK it failed with error in alignment/size restrictions for DPDK's internal heap. I was able to hack DPDK code to return 2MB aligned addresses for its internal heap and this worked.

So, to confirm my understanding here: for SPDK to work without hugepages, just a guarantee that (2MB)aligned addresses are returned by DPDK's internal heap should be enough?