Re: [RFC 26/26] plugins/amdgpu: Hack to work around GPU virtual address conflicts

Patches 22, 23, and 24 are
This patch is
Reviewed-By: David Francis <[email protected]>

Patches 21, 25, and 26 somewhat depend on what the future plan is for this feature.
Full support of no-kfd amdgpu with CRIU is going to involve changes to the kernel
interfaces that will replace the code added in these patches. If these are allowing real
workloads to complete that would otherwise fail, go ahead. But if they are meant to
be first drafts, I might want to wait until the work can be done properly.

David

________________________________________
From: Tvrtko Ursulin <[email protected]>
Sent: Friday, February 20, 2026 7:05 AM
To: [email protected]
Cc: Francis, David; Tvrtko Ursulin
Subject: [RFC 26/26] plugins/amdgpu: Hack to work around GPU virtual address conflicts

During the restore process buffer objects are created and mapped into the
GPU virtual address space to their final addresses, and then their content
is attempted to be restored.

This step is done by instantiating a libdrm instance and the associated
userspace virtual address range manager. This manager is unaware of the
previous address range allocations on the same file, which in practice
means restore will simply fail.

To work around it we add some retries to try different addresses a few
times, but for a proper solution we will need to either add a mechanism
for reserving ranges in libdrm va manager, or rework the order of the
restore operations so that the buffer objects are only mapped into their
final locations once their content has been restored.

Signed-off-by: Tvrtko Ursulin <[email protected]>
---
 plugins/amdgpu/amdgpu_plugin.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/plugins/amdgpu/amdgpu_plugin.c b/plugins/amdgpu/amdgpu_plugin.c
index 239eeebc5111..b2f192a80f65 100644
--- a/plugins/amdgpu/amdgpu_plugin.c
+++ b/plugins/amdgpu/amdgpu_plugin.c
@@ -573,6 +573,7 @@ int sdma_copy_bo(int shared_fd, uint64_t size, int storage_fd,
        amdgpu_context_handle h_ctx;
        uint32_t *ib = NULL;
        int j, err, err2, packets_per_buffer;
+       unsigned int retries = 0;

        buffer_bo_size = min(size, buffer_size);
        packets_per_buffer = ((buffer_bo_size - 1) / max_copy_size) + 1;
@@ -603,6 +604,8 @@ int sdma_copy_bo(int shared_fd, uint64_t size, int storage_fd,
                return -EINVAL;
        }

+retry_src_va:
+       retries++;
        err = amdgpu_va_range_alloc(h_dev, amdgpu_gpu_va_range_general, src_bo_size, 0x1000, 0, &gpu_addr_src,
                                    &h_va_src, 0);
        if (err) {
@@ -611,6 +614,8 @@ int sdma_copy_bo(int shared_fd, uint64_t size, int storage_fd,
        }
        err = amdgpu_bo_va_op(h_bo_src, 0, src_bo_size, gpu_addr_src, 0, AMDGPU_VA_OP_MAP);
        if (err) {
+               if (retries < 1000)
+                       goto retry_src_va;
                pr_perror("failed to GPU map the src BO");
                goto err_src_bo_map;
        }
@@ -639,6 +644,9 @@ int sdma_copy_bo(int shared_fd, uint64_t size, int storage_fd,
                goto err_dst_bo_prep;
        }

+       retries = 0;
+retry_dst_va:
+       retries++;
        err = amdgpu_va_range_alloc(h_dev, amdgpu_gpu_va_range_general, dst_bo_size, 0x1000, 0, &gpu_addr_dst,
                                    &h_va_dst, 0);
        if (err) {
@@ -647,6 +655,8 @@ int sdma_copy_bo(int shared_fd, uint64_t size, int storage_fd,
        }
        err = amdgpu_bo_va_op(h_bo_dst, 0, dst_bo_size, gpu_addr_dst, 0, AMDGPU_VA_OP_MAP);
        if (err) {
+               if (retries < 1000)
+                       goto retry_dst_va;
                pr_perror("failed to GPU map the dest BO");
                goto err_dst_bo_map;
        }
--
2.52.0