[SPDK] Re: vhost is destroyed many times when starting vhost device

Hi Feng,

The issue you mentioned does exist, but it's not related with number of disk, but with "num-queues" parameter input from
QEMU, in my test environment, if I used 8 IO queues, the vhost_blk_start() will be called 9 times.  Note that the BIOS will
use 1 IO queue so vhost_blk_start() is called 1 time during the BIOS stage.

In the vhost target side, we don't know the exactly number of queues used for this device, so we will start the device once
got one valid IO queue, and restart the device when got a new IO queue.  That's the reason why the start/stop device will
be called multiple times.

> -----Original Message-----
> From: Li Feng <fengli(a)smartx.com>
> Sent: Thursday, April 23, 2020 8:30 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] vhost is destroyed many times when starting vhost device
> 
> Hi,
> 
> In rte_vhost_compat.c, spdk_extern_vhost_pre_msg_handler will call
> destroy_device when some conditions meet.
> 
> I tested and found when a vhost-blk setup, the vhost device will be
> destroyed 9 times.
> And the bdev(no matter which type) will be affected. I'm using the aio
> bdev, and the io_setup/io_destroy will be called 9 times when a disk
> is inserted to a VM.
> 
> My concern is:
> Is there any solution to reduce the call times(at least: disks nums * 9)?
> The call will slow down the VM boot time if VM has multiple disks.
> There are some  related logs:
> 
> Starting SPDK v20.01-pre git sha1 93eecf627 / DPDK 19.11.0 initialization...
> [ DPDK EAL parameters: spdk_tgt --no-shconf -c 1 -m 1
> --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6
> --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations
> --file-prefix=spdk_pid2004865 ]
> EAL: No available hugepages reported in hugepages-1048576kB
> app.c: 645:spdk_app_start: *NOTICE*: Total cores available: 1
> reactor.c: 346:_spdk_reactor_run: *NOTICE*: Reactor started on core 0
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 174:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 193:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> destroy device
> rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> 
> Thanks,
> 
> Feng Li
> 
> --
> The SmartX email address is only for business purpose. Any sent message
> that is not related to the business is not authorized or permitted by
> SmartX.
> 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的
> 邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.
> 
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org