[PATCH v3 2/3] migration/rdma: Post initial receive before accepting
"Yanfei Xu" <[email protected]>
| Newsgroups | org.nongnu.qemu-devel |
|---|---|
| Message-ID | <[email protected]> |
The source return-path thread can send READY before the destination has
posted its initial receive. Since RNR retries default to zero, the send
can fail and abort migration.
Post the initial receive before rdma_accept() to close this window.
Adding delays can easily reproduce it. For exmaple with sleep(1), error
could be found at the source side:
2026-08-20T05:55:11.471421Z qemu_rdma_exchange_get_response_start CONTROL: READY receiving...
2026-08-20T05:55:11.471422Z qemu_rdma_post_send_control CONTROL: sending QEMU FILE..
2026-08-20T05:55:11.471447Z qemu_rdma_poll_other other completion 2000 received left 0
2026-08-20T05:55:11.471589Z qemu_rdma_poll_recv completion 0 received (4000) left 0
2026-08-20T05:55:11.471592Z qemu_rdma_poll_other other completion 4000 received left 0
2026-08-20T05:55:11.471594Z qemu_rdma_exchange_get_response_start CONTROL: READY receiving...
2026-08-20T05:55:11.471596Z qemu_rdma_post_send_control CONTROL: sending QEMU FILE..
2026-08-20T05:55:11.471617Z qemu_rdma_poll_other other completion 2000 received left 0
2026-08-20T05:55:11.705561Z qemu_rdma_close
2026-08-20T05:55:11.705587Z qemu_rdma_close
2026-08-20T05:55:11.705626Z qemu-system-x86_64: rdma migration: send polling control error
2026-08-20T05:55:11.707001Z qemu-system-x86_64: warning: Early error. Sending error.
2026-08-20T05:55:11.707031Z qemu_rdma_post_send_control CONTROL: sending ERROR..
2026-08-20T05:55:11.707038Z qemu-system-x86_64: warning: rdma migration: send polling control error
2026-08-20T05:55:11.707301Z qemu_rdma_cleanup_disconnect
2026-08-20T05:55:11.710569Z qemu_rdma_cleanup_disconnect
Fixes: 55cc1b5937a8 ("migration: create a dedicated connection for rdma return path")
Signed-off-by: Yanfei Xu <[email protected]>
Reviewed-by: Peter Xu <[email protected]>
Reviewed-by: Jack Wang <[email protected]>
---
migration/rdma.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/migration/rdma.c b/migration/rdma.c
index fc6cb75167..b79f4263f7 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3274,6 +3274,12 @@ static int qemu_rdma_accept(RDMAContext *rdma)
}
}
+ ret = qemu_rdma_post_recv_control(rdma, RDMA_WRID_READY, &err);
+ if (ret < 0) {
+ error_report_err(err);
+ goto err_rdma_dest_wait;
+ }
+
/* Accept the second connection request for return path */
if ((migrate_postcopy() || migrate_return_path())
&& !rdma->is_return_path) {
@@ -3306,12 +3312,6 @@ static int qemu_rdma_accept(RDMAContext *rdma)
rdma_ack_cm_event(cm_event);
rdma->connected = true;
- ret = qemu_rdma_post_recv_control(rdma, RDMA_WRID_READY, &err);
- if (ret < 0) {
- error_report_err(err);
- goto err_rdma_dest_wait;
- }
-
qemu_rdma_dump_gid("dest_connect", rdma->cm_id);
return 0;
--
2.20.1