[SPDK] Re: For Dummies guide to setting up ramdisk NVMe/TCP target

Hi Eric,

Yeah, run with sudo.  I get the same thing when I run w/o, it’s DPDK unable to get the PA with privilege.

Thx
Paul

From: Luse, Paul E <paul.e.luse(a)intel.com>
Date: Wednesday, October 14, 2020 at 4:43 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>, Rui Chang <Rui.Chang(a)arm.com>
Cc: spdk(a)lists.01.org <spdk(a)lists.01.org>
Subject: Re: [SPDK] Re: For Dummies guide to setting up ramdisk NVMe/TCP target
Hi Eric,

Bummer that’s not working.  It works OK for my with vbox 6.1 on a mac and ubuntu 1901 and the output says iova-mode=pa. Did you change any settings on your VM and try simple stuf like adding memory and/or running with one core (how many cores did you give your VM?).  Also are you running privelaged?

Thx
Paul

peluse(a)pels:~/spdk$ sudo ./build/bin/nvmf_tgt
[2020-10-14 23:38:50.539538] Starting SPDK v20.10-pre git sha1 779a6bdf8 / DPDK 20.08.0 initialization...
[2020-10-14 23:38:50.540128] [ DPDK EAL parameters: [2020-10-14 23:38:50.540346] nvmf [2020-10-14 23:38:50.540556] --no-shconf [2020-10-14 23:38:50.540755] -c 0x1 [2020-10-14 23:38:50.540957] --log-level=lib.eal:6 [2020-10-14 23:38:50.541153] --log-level=lib.cryptodev:5 [2020-10-14 23:38:50.541342] --log-level=user1:6 [2020-10-14 23:38:50.541566] --iova-mode=pa [2020-10-14 23:38:50.541763] --base-virtaddr=0x200000000000 [2020-10-14 23:38:50.541957] --match-allocations [2020-10-14 23:38:50.542290] --file-prefix=spdk_pid918 [2020-10-14 23:38:50.542544] ]
EAL: No legacy callbacks, legacy socket not created
[2020-10-14 23:38:50.552172] app.c: 667:spdk_app_start: *NOTICE*: Total cores available: 1

From: Curtin, Eric <Eric.Curtin(a)dell.com>
Date: Wednesday, October 14, 2020 at 1:56 AM
To: Rui Chang <Rui.Chang(a)arm.com>
Cc: spdk(a)lists.01.org <spdk(a)lists.01.org>
Subject: [SPDK] Re: For Dummies guide to setting up ramdisk NVMe/TCP target
How would I get my CPU mask on Linux? I'm running on a Virtualbox VM. I also used 0xf as a CPU mask and this was my output:

./build/bin/nvmf_tgt -m 0xf
[2020-10-14 09:47:07.650399] Starting SPDK v20.10-pre git sha1 779a6bdf8 / DPDK 20.08.0 initialization...
[2020-10-14 09:47:07.651075] [ DPDK EAL parameters: [2020-10-14 09:47:07.651092] nvmf [2020-10-14 09:47:07.651126] --no-shconf [2020-10-14 09:47:07.651132] -c 0xf [2020-10-14 09:47:07.651138] --log-level=lib.eal:6 [2020-10-14 09:47:07.651144] --log-level=lib.cryptodev:5 [2020-10-14 09:47:07.651150] --log-level=user1:6 [2020-10-14 09:47:07.651160] --iova-mode=pa [2020-10-14 09:47:07.651166] --base-virtaddr=0x200000000000 [2020-10-14 09:47:07.651172] --match-allocations [2020-10-14 09:47:07.651181] --file-prefix=spdk_pid702834 [2020-10-14 09:47:07.651187] ]
EAL: FATAL: Cannot use IOVA as 'PA' since physical addresses are not available
EAL: Cannot use IOVA as 'PA' since physical addresses are not available
[2020-10-14 09:47:07.659984] init.c: 577:spdk_env_init: *ERROR*: Failed to initialize DPDK
[2020-10-14 09:47:07.659999] app.c: 535:app_setup_env: *ERROR*: Unable to initialize SPDK env

Regards,

Eric Curtin
Software Engineer

Ovens Campus,
Cork,
Ireland

Dell EMC

Next planned absences: NA
________________________________
From: Rui Chang <Rui.Chang(a)arm.com>
Sent: Wednesday 14 October 2020 08:50
To: Curtin, Eric <Eric.Curtin(a)dell.com>
Cc: spdk(a)lists.01.org <spdk(a)lists.01.org>
Subject: RE: [SPDK] For Dummies guide to setting up ramdisk NVMe/TCP target


[EXTERNAL EMAIL]

Hi Eric



Refer to this page for HOWTO:

https://spdk.io/doc/nvmf.html



You said you want to run on one machine, suppose your machine’s IP is 8.8.8.6



  1.  Build your SPDK

git clone https://github.com/spdk/spdk spdk

cd spdk

git submodule update --init

./configure

make



  1.  Run:



Start target:

                ./build/bin/nvmf_tgt -m 0xf          (-m gives the CPU mask. Normally you can make it on the same CPU socket as network card to get better performance)



On another console, run rpc comamnds to do configuration:

./scripts/rpc.py nvmf_create_transport -t TCP      (there are lots of additional options you can use if you wish)

./scripts/rpc.py bdev_malloc_create -b Malloc0 200 512

./scripts/rpc.py nvmf_create_subsystem nqn.2016-06.io.spdk:cnode1 -a -s SPDK00000000000001 -d SPDK_Controller1

./scripts/rpc.py nvmf_subsystem_add_ns nqn.2016-06.io.spdk:cnode1 Malloc0

./scripts/rpc.py nvmf_subsystem_add_listener nqn.2016-06.io.spdk:cnode1 -t TCP -a 8.8.8.6 -s 4420                 (suppose 8.8.8.6 is the IP you want the target to listen on)



Run io:

./build/examples/perf -r 'trtype:TCP adrfam:IPv4 traddr:8.8.8.6 trsvcid:4420 subnqn:nqn.2016-06.io.spdk:cnode1'  -q 128 -o 4096  -w write -t 100 -c 0xf0  (-c give the cpu mask for perf tool, you’d better make it different with nvmf_tgt to get better performance)



Regards,

Rui





-----Original Message-----

From: eric.curtin(a)dell.com<mailto:eric.curtin(a)dell.com> <eric.curtin(a)dell.com<mailto:eric.curtin(a)dell.com>>

Sent: Thursday, October 1, 2020 3:23 PM

To: spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>

Subject: [SPDK] For Dummies guide to setting up ramdisk NVMe/TCP target



Hi Guys,



I'm just getting started with SPDK and wanted to create a fake ramdisk NVMe/TCP target block device, just so I could simulate NVMe/TCP traffic on a single Linux instance for development purposes.



Anybody know how to set up a machine like this for NVMe/TCP development?

_______________________________________________

SPDK mailing list -- spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>

To unsubscribe send an email to spdk-leave(a)lists.01.org<mailto:spdk-leave(a)lists.01.org>

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org