[SPDK] [RFC] Introduction of DAOS bdev

Denis Barahtanov <denis.barahtanov at croit.io> Thu, 14 Apr 2022 11:56:02 +1000
Newsgroups dev.linux.lists.spdk
Message-ID <CABHopajQCqFaFB21d0xRThG_HSd47ZSXTmmr7sYHuWv-Z=QOuw@mail.gmail.com>
Hello,

We are working on a new bdev type that leverages DAOS DFS as a backend
storage (https://github.com/daos-stack/daos).

The patch is on gerrit: https://review.spdk.io/gerrit/c/spdk/spdk/+/12260

Design wise this bdev is a file named as the bdev itself in the DAOS POSIX
container that uses daos event queue per io channel.
Having an event queue per io channel is showing the best IO throughput.
The implementation supports sharing pools and containers connection between
devices for better connection usage.

The semantic of usage is the same as any other bdev type, to build spdk
with daos support daos-devel package has to be installed:

$ ./configure --with-daos

To run it, the target machine should have daos_agent up and running, as
well as the pool and POSIX container ready to use.
And then to export bdev over tcp:

$ ./nvmf_tgt -m [21-24] &
$ ./scripts/rpc.py nvmf_create_transport -t TCP -u 2097152 -i 2097152

$ ./scripts/rpc.py bdev_daos_create -b daosdev0 -p pool-label -c cont-label
1048576 4096

$ ./scripts/rpc.py nvmf_create_subsystem nqn.2016-06.io.spdk1:cnode1 -a -s
SPDK00000000000001 -d SPDK_Virtual_Controller_1
$ ./scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk1:cnode1
daosdev0
$ ./scripts/rpc.py nvmf_subsystem_add_listener nqn.2016-06.io.spdk1:cnode1
-t tcp -a <IP> -s 4420


On the initiator side, make sure that `nvme-tcp` module is loaded:

$ nvme connect-all -t tcp -a 172.31.91.61 -s 4420

$ nvme list
Node             SN                   Model
   Namespace Usage                      Format           FW Rev
---------------- --------------------
---------------------------------------- ---------
-------------------------- ---------------- --------
/dev/nvme8n1     SPDK00000000000001   SPDK_Virtual_Controller_1
   1           1.10  TB /   1.10  TB      4 KiB +  0 B   22.05


Looking forward to any suggestions.

Best regards,
Denis.