[SPDK] May NVMe-oF Target be extended into a data-switch?
330416470 at qq.com Thu, 23 Sep 2021 03:39:16 +0000
| Newsgroups | dev.linux.lists.spdk |
|---|---|
| Message-ID | <[email protected]> |
This is an idea during development, may it be feasible? In NVMe-oF Target framework, data is received from RDMA or tcp channel, then written into NVMe SSD. NVMe-oF Target transport layer is closed and can not be accessed by other modules. So it is not ok for strorage cluster, and data must be copied from source transport to destination transport when transmitting data from mater to slave nodes. To solve this, transport layer's data pool will be expose and shared between RDMA and NVMe channels, which makes whole framework confused. To be more comprehensive and acceptable, maybe NVMe-oF Target can be extended to a data switch, in witch receiving data from one channel and transmitting to one or multi channels without copying. The switch can support many interface, such as tcp buffer, RDMA queue, Nvme channel, Vhost vring, and also host memory buffer. Each datapath can have single input and one or multiple outputs, that is decided by APP service. NVMe-oF Target is only one application scene for data switch. This software data switch is suitable for storage cluster and vfio-user, vhost-user. The difficulty is memory must be registered into all modules, such as RNIC and PCIe, and the memory should be shared among multiple modules. There are a lot of work to be done.