Re: Steps towards live migration

+CC [John & Joerg]

Hi Jacob,

I briefly looked at your code. I have some questions below to understand 
your design more. Do you have a comprehensive design document somewhere?

> Hi,
> 
> as mentioned during the last SVSM development talk, I am currently writing my
> thesis on live migration of confidential guests. With this email I would like to
> start a discussion so that the same problem is not addressed multiple times

Ofcourse. That's the reason we shared our design in the kvm forum talk 
and asked for collaboration on common components (open vendor agnostic 
problems):

https://kvm-forum.qemu.org/2024/SNP_Live_Migration_KVM_forum_2024_svDwxa3.pdf


> independently. I also welcome any feedback to make the code written useful for
> SVSM Coconut (not only for the thesis). Development is done on machine with
> SEV-SNP support. I made a pull request with the SVSM patches[1] to ease the
> discussion.
> 
> Here is a summary of progress to date:
> 
> All validated guest pages can be transferred from the source SVSM to the target
> SVSM. Currently, no packaging (no confidentiality) is performed, but a hash of
> all transferred/received blocks is computed that can be compared to verify that
> the channel is working correctly. Also, the main migration function of the

Any analysis on algorithm you plan to use for this? IIUC you are 
currently sharing the hash from source to destination? Wouldn't the 
'aes-gcm' in [4] will do that in addition to encryption?

> source and destination SVSM is busy waiting for a signal from QEMU to start
> outbound or incoming migration.

 > > A single shared page called MigrationPage is used for data transfer and
> communication with QEMU. The migration page contains two registers and a buffer:
> a status register, a data register, and a data buffer. The status register is
> used to signal a change in status (e.g., migration starts, migration is
> complete). The data register is used to signal that a new page has been prepared
> in the data buffer by the provider or processed by the consumer. The roles of
> provider and consumer are switched between SVSM and QEMU on the source and
> destination machines.

This looks like a communication channel between Qemu and SVSM. We are 
re-using few bits in per CPU ghcb page for SVSM <-> host commands 
communication.

I can understand you are not using additional vCPUs. All this can be too 
much work for guest general purpose vCPUs.

> 
> The QEMU patch[2] implements communication with the SVSM migration handler and
> block transfer from source to destination. The idea is that creating the
> communication channel is all the hypervisor should do, the rest should be done
> in SVSM.
> 
> Current plan for the future in order of realisation:
> 
> (1) A function that puts all hosted vCPUs (except the migration handler) into a

[...]

>      spinning state. The hypervisor is not trusted, so the SVSM must be able to
>      ensure that it is not running any vCPUs. For this task, I consider two-phase
>      checkpointing[3].

Can you please elaborate more this.

> (2) Migrate the machine from the source to the destination with all vCPUs
>      stopped by the function from (1).

You mean black-out phase?

> (3) Secret key establishment - though about using [5].

We need to tackle this problem. A probable collaboration point, maybe 
coupled with attestation and migration key sharing.

> (4) Package the pages - authenticated encryption using [4].
> (5) Dirty page tracking.
> (6) Move the SVSM migration handler on an extra vCPU.
> (7) Start migration handler on signal instead of busy-waiting loop.

There still are other open questions at more granular level. Can discuss 
those as well, once I go through your complete design.

But at higher level, I would like to reuse more of the existing 
functionality in Qemu for live migration and use SVSM for memory 
packaging, guests memory permission setting at VMPL0, and live migration 
sanity related tasks.

Best regards,
Pankaj

> 
> Best regards,
> Jakub
> 
> [1] https://github.com/coconut-svsm/svsm/pull/745
> [2] https://github.com/coconut-svsm/qemu/pull/23
> [3] https://ipads.se.sjtu.edu.cn/_media/publications/sgxmigration-dsn17.pdf
> [4] https://github.com/RustCrypto/AEADs/blob/master/aes-gcm/src/lib.rs
> [5] https://github.com/nihalpasham/static-dh-ecdh