[SPDK] Re: vhost is destroyed many times when starting vhost device

Hi Xiaodong,
Thanks for your comment.

Add `spdk_put_io_channel` in bdev_remove_cb could resolve the hotplug issue.
I have updated the patch.

Feng Li

Liu, Xiaodong <xiaodong.liu(a)intel.com> 于2020年4月25日周六 上午8:47写道：
>
> Hi, Feng
>
> Thanks for your digging on this scenario. Yes, it is really a problem on which we should consider more.
>
> I looked into your patch. But I just think you missed the situation if the backend bdev is hot removed, seems the dummy_io_channel will prevent the removal process.
>
>  --Thanks
> From Xiaodong
>
> -----Original Message-----
> From: Li Feng <fengli(a)smartx.com>
> Sent: Friday, April 24, 2020 10:44 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] Re: vhost is destroyed many times when starting vhost device
>
> Hi,
> I have submitted a patch[1] to partially fix this issue.
> Any suggestions?
> After applying this patch, the log of call io_setup/io_destroy will be gone.
>
> [1]: https://review.spdk.io/gerrit/c/spdk/spdk/+/2020
>
> Thanks,
>
> Feng Li
>
> Li Feng <fengli(a)smartx.com> 于2020年4月23日周四 下午8:29写道：
> >
> > Hi,
> >
> > In rte_vhost_compat.c, spdk_extern_vhost_pre_msg_handler will call
> > destroy_device when some conditions meet.
> >
> > I tested and found when a vhost-blk setup, the vhost device will be
> > destroyed 9 times.
> > And the bdev(no matter which type) will be affected. I'm using the aio
> > bdev, and the io_setup/io_destroy will be called 9 times when a disk
> > is inserted to a VM.
> >
> > My concern is:
> > Is there any solution to reduce the call times(at least: disks nums * 9)?
> > The call will slow down the VM boot time if VM has multiple disks.
> > There are some  related logs:
> >
> > Starting SPDK v20.01-pre git sha1 93eecf627 / DPDK 19.11.0 initialization...
> > [ DPDK EAL parameters: spdk_tgt --no-shconf -c 1 -m 1
> > --log-level=lib.eal:6 --log-level=lib.cryptodev:5 --log-level=user1:6
> > --iova-mode=pa --base-virtaddr=0x200000000000 --match-allocations
> > --file-prefix=spdk_pid2004865 ]
> > EAL: No available hugepages reported in hugepages-1048576kB
> > app.c: 645:spdk_app_start: *NOTICE*: Total cores available: 1
> > reactor.c: 346:_spdk_reactor_run: *NOTICE*: Reactor started on core 0
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 174:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 193:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> > rte_vhost_compat.c: 225:spdk_extern_vhost_pre_msg_handler: *ERROR*:
> > destroy device
> > rte_vhost_compat.c: 130:stop_device: *ERROR*: call stop_device
> > bdev_aio.c: 581:bdev_aio_group_destroy_cb: *ERROR*:  call io_destroy
> > bdev_aio.c: 563:bdev_aio_group_create_cb: *ERROR*:  call io_setup
> >
> > Thanks,
> >
> > Feng Li
>
> --
> The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX.
> 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.
>
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org

-- 
The SmartX email address is only for business purpose. Any sent message 
that is not related to the business is not authorized or permitted by 
SmartX.
本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.