EDK2 CAA Page Fragmented Allocation
Adam Dunlap <[email protected]>
| Newsgroups | dev.linux.lists.coconut-svsm |
|---|---|
| Message-ID | <CAMBK9=YfWzsXqhav8d8-cLtn6RKLBKY4PN2FCHwjCci1Mi-fhQ@mail.gmail.com> |
Hi all, I've been debugging a boot failure on VMs running under Coconut-SVSM with 120 or more vCPUs. The guest kernel (on several distros) is hanging while trying to allocate a 256MiB memblock out of "low" memory for the SWIOTLB. While debugging this, I noticed that the memory map reported by EDK2 has a ton of entries. There are the expected reserved regions, but then there's a section of about 2x #vcpus where there is 1 reserved and 1 usable page alternating. It turns out the reserved pages are allocated as CAA pages and then usable pages are allocated (and then freed) as VMSA pages at this[1] point in EDK2. Note that while these pages are allocated many times over the various UEFI phases, it's only the CAA pages from the first allocation that are getting leaked. I had a few questions that I thought someone here might know the answer to: 1. Is UEFI supposed to keep these CAA pages allocated? I believe that UEFI is supposed to be able to talk to the TPM post-ExitBootServices and that would likely require CAA pages, but I might be missing something. 2. How are the other CAA pages being freed? The commit that adds the allocation [2] does not add any corresponding frees 3. If everything is working pretty much as-expected, then it'd probably be a good idea to allocate all the CAA pages in one big block so that they just take up one big entry in the memory map. Obviously a large/fragmented memory map *shouldn't* cause the guest kernel to hang, but we have hit issues in the past related to this, and blowing up the memory map from having ~10 entries to having several hundred does not seem good. Please let me know if this memory map pattern appears on other platforms (maybe I just messed up the edk2 backports) or if you have any other thoughts about what might be happening. [1] https://github.com/coconut-svsm/edk2/blob/69d7c826dde066d389e0ba76112b75f4259584eb/UefiCpuPkg/Library/MpInitLib/X64/AmdSev.c#L144 [2] https://github.com/coconut-svsm/edk2/commit/a010681f74c9110a9e8e1753a98c13743b60f7f7 Thanks! Adam