[SPDK] NVMe Multipath subsystem controller IDs

Jonathan Teh <jonathan.teh at mayadata.io> Thu, 01 Apr 2021 10:49:01 +0100
Newsgroups dev.linux.lists.spdk
Message-ID <[email protected]>
I'm looking at using Linux's native NVMe multipathing with SPDK's nvmf 
target where multiple SPDK instances are used.

As the subsystem issues controller IDs in the range [1, 0xFFEF], the 
initial connection to each individual target ends up with controller ID 
1, leading to a clash for the connection to the 2nd, and subsequent, 
targets requiring multiple connections to "cycle" to the non-clashing ID.

This issue that was previously discussed in 
https://lists.01.org/hyperkitty/list/spdk(a)lists.01.org/thread/ADLA6IPHVGV5LW2AXZUBWLCYQW3HGOND/ 
but I don't think a resolution was agreed upon.

One approach would be to limit the controller ID range per subsystem 
(which was also previously proposed) which was implemented for the Linux 
kernel NVMe target in 5.7 as 
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=94a39d61f80fcd679debda11e1ca02b88d90e67e. 


Would you consider a pull request to implement a similar feature in 
SPDK? An external orchestrator could then assign the subsystem in each 
SPDK instance a non-overlapping controller ID range.

Thanks.