[SPDK] Re: Spdk nvme CLI - block device

> -----Original Message-----
> From: Gili Buzaglo <Gili.Buzaglo(a)kioxia.com>
> Sent: Monday, April 6, 2020 9:45 AM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Re: Spdk nvme CLI - block device
> 
> Hi
> Thanks for your reply..
> The goal is to have an initiator in user space which connects to nvmf target and
> exposes block devices in the host(initiator) machine.
> Same way nvme-cli does it... But without the Linux kernel driver in the initiator.
> The last limitation derives from the fact that not all customer would move to a
> kernel which supports tcp nvmf

You can use NBD for this. The setup won't be entirely intuitive, but you can do this:

User space application --(POSIX/libaio)--> Linux kernel NBD device -> user space nvmf target "proxy" - -(network)--> real nvmf target

You end up running the SPDK target on both the initiator and the target, where the one on the initiator is acting as a proxy between NBD and NVMe-oF. When you set up the proxy, you'd want to create an NVMe-oF bdev first that connects to the real target. Then add an NBD bdev on top of it that exposes the block device to the kernel. There's tests that set up NBD disks in test/bdev/blockdev.sh and test/bdev/nbd_common.sh that can serve as examples.

> 
> Thanks
> -gili
> ________________________________
> From: Zawadzki, Tomasz <tomasz.zawadzki(a)intel.com>
> Sent: Monday, April 6, 2020 6:14:42 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Re: Spdk nvme CLI - block device
> 
> Hello Gili,
> 
> If I understand correctly the goal is to prepare data on a bdev before exposing it
> via NVMe-oF (or other targets - vhost/iSCSI) ?
> 
> One way would be to connect to SPDK target with kernel initiator - even locally
> within single machine.
> Then perform necessary preparation using already available tooling.
> As you've noted it is possible to create device in kernel from bdev using NBD as
> well.
> 
> Please see https://spdk.io/doc/bdev.html and appropriate target documentation
> to help with set of RPCs that might help.
> 
> Note:
> nvme CLI is not needed to configure SPDK target or SPDK initiator, this can be
> done via RPC.
> Branch spdk-1.6 for nvme CLI was created to interact with NVMe bdevs in SPDK.
> Which since has been improved to work with stock nvme CLI when using
> https://spdk.io/doc/bdev.html#bdev_config_nvme_cuse
> 
> Thanks,
> Tomek
> 
> > -----Original Message-----
> > From: Gili Buzaglo <Gili.Buzaglo(a)kioxia.com>
> > Sent: Saturday, April 4, 2020 11:59 AM
> > To: spdk(a)lists.01.org
> > Subject: [SPDK] Spdk nvme CLI - block device
> >
> > Hi
> > I tried to understand if nvme CLI based on spdk-1.6 can somehow
> > generate a block device without using the nvme fabrics kernel module.
> > I managed to add tcp support and connect ,but then the code goes to
> > the kernel module and no block device was added.
> > In general , I know that it is possible to create block devices which
> > are implemented in user space (for example with nbd) .. and I know
> > spdk has block device layer..
> > Does someone think that what I did is somehow possible ?
> >
> > The motivation for this is easy host installation without the
> > requirement to replace kernel or application changes.
> > I'm aware there will be a performance impact..
> >
> > Thanks
> > -gili
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> > spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org