[SPDK] Re: Configuring SPDK NVMe over TCP initiator

 Or is this configuration using Kernel mode TCP socket itself? If so, any steps as to how to integrate with the user mode TCP stack.
Regards,Chethan
    On Monday, January 25, 2021, 01:39:56 AM GMT+5:30, Chethan Seshadri <chethan_seshadri(a)yahoo.com> wrote:  
 
  Okay, so with the NVMe over TCP target configured on one VM and with two Malloc bdevs configured, I was able to discover them on the initiator size with the steps that Tomasz mentioned (along with creating the TCP transport). Thanks.
I guess I was referring to two things w.r.t user mode TCP stack...
   
   - The NVMe over TCP transport that SPDK uses/creates. I guess lib/nvme/nvme_tcp.c or lib/nvmf/tcp.c points to it.
   - The user mode TCP stack - I guess I was referring to the DPDK library files dpdk/lib/librte_gro/gro_tcp4.c or dpdk/lib/librte_gso/gso_tcp4.c. If these are not about the user mode TCP stack, then can you let me know how do I exercise any user mode TCP stack for this purpose

Thanks,Chethan 
    On Saturday, January 23, 2021, 12:01:58 AM GMT+5:30, Luse, Paul E <paul.e.luse(a)intel.com> wrote:  
 
 This question jumped out at me:

“> Also, does anyone know if I can use the user mode TCP stack bundled along
> with SPDK for regular TCP communication?”

Just to be clear there is no user mode TCP stack bundled with SPDK.  I think you meant the SPDK NVMe-OF initiator which is not “general purpose” as Tomek was explaining requires use of an SPDK application on the initiator side as highlighted below.

Hope that helps.

Thx
Paul

From: Zawadzki, Tomasz <tomasz.zawadzki(a)intel.com>
Date: Friday, January 22, 2021 at 11:26 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Re: Configuring SPDK NVMe over TCP initiator
Can you please describe how do you want to use the device on initiator side ?

rpc.py communicates with a running SPDK application instance on local host. So please make sure that an SPDK application you want to utilize on initiator side is running first.

On initiator start ./build/bin/spdk_tgt and issue the previously mentioned RPC. After it successfully connects to the NVMe-oF target,  'rpc.py bdev_get_bdevs' will display the connected bdev.

Similar things can be achieved with other SPDK applications like bdevperf or fio_plugin by using JSON configuration to those tools.

Thanks,
Tomek

> -----Original Message-----
> From: Chethan Seshadri <chethan_seshadri(a)yahoo.com>
> Sent: Friday, January 22, 2021 5:30 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Re: Configuring SPDK NVMe over TCP initiator
>
>  Thank you Michael and Tomasz.
> While I am able to setup the NVMe over TCP target successfully, for the
> initiator, I executed scripts/setup.sh followed by creating the NVMe device
> of NVMe over TCP resource using the bdev_nvme_attach_controller rpc
> method. But it fails at line # 71 of scripts/rpc/client.py which is basically
> connection establishment error because "Name or service not known". To
> the same NVMe over TCP target, I am able to establish a connection using nc
> i.e.#nc -z -v 10.10.10.1 4420 Do I have to execute any other steps other than
> scripts/setup.sh on the NVMe over TCP initiator side?
> Also, does anyone know if I can use the user mode TCP stack bundled along
> with SPDK for regular TCP communication?
> Thanks,Chethan
>    On Friday, January 22, 2021, 07:40:21 AM GMT+5:30, Michael Haeuptle
> <michaelhaeuptle(a)gmail.com> wrote:
>
>  Hi Chethan,
>
> take a look at https://spdk.io/doc/nvmf.html
> <https://spdk.io/doc/nvmf.html> for setting up the NVMF and
> https://spdk.io/doc/bdev.html#bdev_config_nvme
> <https://spdk.io/doc/bdev.html#bdev_config_nvme> for setting up an
> NVMe bdev.
>
> The basic steps are something like below (haven’t had a chance to verify
> these steps so they may need some adjustments):
>
> Follow https://spdk.io/doc/system_configuration.html
> <https://spdk.io/doc/system_configuration.html> to set up the
> environment. Basically, you have to run scripts/setup.sh and ensure that the
> NVMe device is using the vfio-pci or uio_pci_generic (lspci -vv to verify or run
> the script again to give status).
>
> Then create a bdev for the NVMe device and add it as a namespace to a
> NVMF subsystem:
>
> scripts/rpc.py bdev_nvme_attach_controller -b NVMe1 -t PCIe -a
> 0000:01:00.0
>
> scripts/rpc.py nvmf_create_transport -t TCP -u 16384 -m 8 -c 8192
> scripts/rpc.py nvmf_create_subsystem nqn.2016-06.io.spdk:cnode1 -a -s
> SPDK00000000000001 -d SPDK_Controller1 scripts/rpc.py
> nvmf_subsystem_add_ns nqn.2016-06.io.spdk:cnode1 NVMe1 scripts/rpc.py
> nvmf_subsystem_add_listener nqn.2016-06.io.spdk:cnode1 -t rdma -a
> 192.168.100.8 -s 4420
>
> — Michael
>
> > On Jan 21, 2021, at 5:08 AM, Chethan Seshadri
> <chethan_seshadri(a)yahoo.com> wrote:
> >
> > Hi,
> > Can anyone point me to the steps to configure SPDK NVMe over TCP
> initiator?
> > Thanks,Chethan
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> > spdk-leave(a)lists.01.org
>
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org