[SPDK] Re: "nvmf_create_transport" fails with 16 cores

Oh gotcha ☺  Sure, it’s a holiday in the US today and I’m on my way out the door.  If you don’t get an answer by the time I logon tomorrow morning I’ll do that and see what happens on this end.  Thanks!!

-Paul

From: Charlie Li <charlie.li(a)gmail.com>
Date: Monday, January 18, 2021 at 2:11 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Re: "nvmf_create_transport" fails with 16 cores
Thanks Paul,
I have more than 16 cores. I don't have any problem starting the target
with 16 or more cores.
But when I try to create a TCP transport after the target is started, it
fails. Can you try "sudo ./scripts/rpc.py nvmf_create_transport -t TCP" on
your system?

Charlie


On Mon, Jan 18, 2021 at 2:57 PM Luse, Paul E <paul.e.luse(a)intel.com> wrote:

> This might sound like a dumb question but how many physical cores do you
> have? I just fire up the target on 32 w/o problems:
>
> sudo ./build/bin/nvmf_tgt -m 0xffffffff
>
> …
>
> [2021-01-18 15:56:41.042423] app.c: 536:spdk_app_start: *NOTICE*: Total
> cores available: 32
> [2021-01-18 15:56:41.205684] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 1
> [2021-01-18 15:56:41.206424] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 2
> [2021-01-18 15:56:41.206963] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 3
> [2021-01-18 15:56:41.207589] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 4
> [2021-01-18 15:56:41.208210] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 5
> [2021-01-18 15:56:41.208828] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 6
> [2021-01-18 15:56:41.209481] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 7
> [2021-01-18 15:56:41.210092] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 8
> [2021-01-18 15:56:41.210723] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 9
> [2021-01-18 15:56:41.211361] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 10
> [2021-01-18 15:56:41.212071] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 11
> [2021-01-18 15:56:41.212882] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 12
> [2021-01-18 15:56:41.213586] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 13
> [2021-01-18 15:56:41.214310] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 14
> [2021-01-18 15:56:41.215053] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 15
> [2021-01-18 15:56:41.215987] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 16
> [2021-01-18 15:56:41.216847] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 17
> [2021-01-18 15:56:41.217799] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 18
> [2021-01-18 15:56:41.218646] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 19
> [2021-01-18 15:56:41.219347] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 20
> [2021-01-18 15:56:41.220023] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 21
> [2021-01-18 15:56:41.220869] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 22
> [2021-01-18 15:56:41.221797] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 23
> [2021-01-18 15:56:41.222801] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 24
> [2021-01-18 15:56:41.223822] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 25
> [2021-01-18 15:56:41.224711] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 26
> [2021-01-18 15:56:41.225398] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 27
> [2021-01-18 15:56:41.226399] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 28
> [2021-01-18 15:56:41.227232] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 29
> [2021-01-18 15:56:41.228311] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 30
> [2021-01-18 15:56:41.229336] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 31
> [2021-01-18 15:56:41.230389] reactor.c: 699:reactor_run: *NOTICE*: Reactor
> started on core 0
> [2021-01-18 15:56:41.231103] accel_engine.c:
> 692:spdk_accel_engine_initialize: *NOTICE*: Accel engine initialized to use
> software engine.
>
> From: Charlie Li <charlie.li(a)gmail.com>
> Date: Monday, January 18, 2021 at 1:47 PM
> To: spdk(a)lists.01.org <spdk(a)lists.01.org>
> Subject: [SPDK] "nvmf_create_transport" fails with 16 cores
> Hello,
>
> I am using SPDK 20.10 - "Starting SPDK v20.10 git sha1 e5d26ecc2 / DPDK
> 20.08.0 initialization..."
>
> If I start SPDK NVMe-oF target with 15 cores - "sudo ./build/bin/nvmf_tgt
> -m 0x7FFF",
> "sudo ./scripts/rpc.py nvmf_create_transport -t TCP" works fine.
>
> If I start SPDK NVMe-oF target with 16 cores - "sudo ./build/bin/nvmf_tgt
> -m 0xFFFF",
> "sudo ./scripts/rpc.py nvmf_create_transport -t TCP" fails with the
> following message
>
> request:
> {
>   "trtype": "TCP",
>   "c2h_success": true,
>   "no_wr_batching": false,
>   "method": "nvmf_create_transport",
>   "req_id": 1
> }
> Got JSON-RPC error response
> response:
> {
>   "code": -32603,
>   "message": "Transport type 'TCP' create failed"
> }
>
> Is there any way to use 16 or more cores?
>
> Thanks,
> Charlie
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org