EDK2 CAA Page Fragmented Allocation

Adam Dunlap <[email protected]>
Newsgroups dev.linux.lists.coconut-svsm
Message-ID <CAMBK9=YfWzsXqhav8d8-cLtn6RKLBKY4PN2FCHwjCci1Mi-fhQ@mail.gmail.com>
Hi all,

I've been debugging a boot failure on VMs running under Coconut-SVSM
with 120 or more vCPUs. The guest kernel (on several distros) is
hanging while trying to allocate a 256MiB memblock out of "low" memory
for the SWIOTLB.

While debugging this, I noticed that the memory map reported by EDK2
has a ton of entries. There are the expected reserved regions, but
then there's a section of about 2x #vcpus where there is 1 reserved
and 1 usable page alternating. It turns out the reserved pages are
allocated as CAA pages and then usable pages are allocated (and then
freed) as VMSA pages at this[1] point in EDK2. Note that while these
pages are allocated many times over the various UEFI phases, it's only
the CAA pages from the first allocation that are getting leaked.

I had a few questions that I thought someone here might know the answer to:
1. Is UEFI supposed to keep these CAA pages allocated? I believe that
UEFI is supposed to be able to talk to the TPM post-ExitBootServices
and that would likely require CAA pages, but I might be missing
something.
2. How are the other CAA pages being freed? The commit that adds the
allocation [2] does not add any corresponding frees
3. If everything is working pretty much as-expected, then it'd
probably be a good idea to allocate all the CAA pages in one big block
so that they just take up one big entry in the memory map.

Obviously a large/fragmented memory map *shouldn't* cause the guest
kernel to hang, but we have hit issues in the past related to this,
and blowing up the memory map from having ~10 entries to having
several hundred does not seem good.

Please let me know if this memory map pattern appears on other
platforms (maybe I just messed up the edk2 backports) or if you have
any other thoughts about what might be happening.

[1] https://github.com/coconut-svsm/edk2/blob/69d7c826dde066d389e0ba76112b75f4259584eb/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c#L144
[2] https://github.com/coconut-svsm/edk2/commit/a010681f74c9110a9e8e1753a98c13743b60f7f7

Thanks!
Adam
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.