[SPDK] [RFC] Introduction of DAOS bdev
Denis Barahtanov <denis.barahtanov at croit.io> Thu, 14 Apr 2022 11:56:02 +1000
| Newsgroups | dev.linux.lists.spdk |
|---|---|
| Message-ID | <CABHopajQCqFaFB21d0xRThG_HSd47ZSXTmmr7sYHuWv-Z=QOuw@mail.gmail.com> |
Hello, We are working on a new bdev type that leverages DAOS DFS as a backend storage (https://github.com/daos-stack/daos). The patch is on gerrit: https://review.spdk.io/gerrit/c/spdk/spdk/+/12260 Design wise this bdev is a file named as the bdev itself in the DAOS POSIX container that uses daos event queue per io channel. Having an event queue per io channel is showing the best IO throughput. The implementation supports sharing pools and containers connection between devices for better connection usage. The semantic of usage is the same as any other bdev type, to build spdk with daos support daos-devel package has to be installed: $ ./configure --with-daos To run it, the target machine should have daos_agent up and running, as well as the pool and POSIX container ready to use. And then to export bdev over tcp: $ ./nvmf_tgt -m [21-24] & $ ./scripts/rpc.py nvmf_create_transport -t TCP -u 2097152 -i 2097152 $ ./scripts/rpc.py bdev_daos_create -b daosdev0 -p pool-label -c cont-label 1048576 4096 $ ./scripts/rpc.py nvmf_create_subsystem nqn.2016-06.io.spdk1:cnode1 -a -s SPDK00000000000001 -d SPDK_Virtual_Controller_1 $ ./scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk1:cnode1 daosdev0 $ ./scripts/rpc.py nvmf_subsystem_add_listener nqn.2016-06.io.spdk1:cnode1 -t tcp -a <IP> -s 4420 On the initiator side, make sure that `nvme-tcp` module is loaded: $ nvme connect-all -t tcp -a 172.31.91.61 -s 4420 $ nvme list Node SN Model Namespace Usage Format FW Rev ---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- /dev/nvme8n1 SPDK00000000000001 SPDK_Virtual_Controller_1 1 1.10 TB / 1.10 TB 4 KiB + 0 B 22.05 Looking forward to any suggestions. Best regards, Denis.