[SPDK] Re: The difference between vhost-nvme and vhost-blk

ok, thanks,
one more follow up question:
isn't this leads to double DMA of data? one from guest to DRAM region and one from DRAM region to device buffer. 











At 2022-03-08 21:31:44, "Thanos Makatos" <thanos.makatos(a)nutanix.com> wrote:
>> -----Original Message-----
>> From: zbhhbz <zbhhbz(a)yeah.net>
>> Sent: 08 March 2022 13:20
>> To: Storage Performance Development Kit <spdk(a)lists.01.org>
>> Subject: [SPDK] Re: The difference between vhost-nvme and vhost-blk
>> 
>> 
>> 
>> 
>> thanks, this helps a lot.
>> 
>> 
>> about the vfio-user,
>> 
>> 
>> I understand that in the vfio-user,
>> the guestOS can issue DMA read/write to a "pcie space " of a virtual device,
>> but I'm confused:
>> 1. does the guestOS issue DMA read/write to region on actual physical device
>> or just a DRAM region?
>
>A DRAM region.
>
>>      if the guestOS directly access the physical device(DMA,IOMMU), where
>> does spdk stands?
>> 2. why do vfio-user need a socket, what kind of data does the socket carries?
>
>The vfio-user protocol allows a device to be emulated outside QEMU (the vfio-user client), in a separate process (the vfio-user server, SPDK running the nvmf/vfio-user target in our case). The UNIX domain socket is used between QEMU and SPDK for initial device setup, virtual IRQs, and other infrequent operations.
>
>> 3. Does the vfio-user look like the vhost-user except for direct DMA access
>> instead of shared memory communication?
>
>vfio-user allows any kind of device to be emulated in a separate process (even non-PCI), while vhost-user is mainly for VirtIO devices.
>
>> 
>> 
>> thanks
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> At 2022-03-08 17:40:12, "Thanos Makatos" <thanos.makatos(a)nutanix.com>
>> wrote:
>> >> -----Original Message-----
>> >> From: zbhhbz <zbhhbz(a)yeah.net>
>> >> Sent: 08 March 2022 09:29
>> >> To: spdk <spdk(a)lists.01.org>
>> >> Subject: [SPDK] Re: The difference between vhost-nvme and vhost-blk
>> >>
>> >> thanks, follow up questions:
>> >> 1. if I still use Vhost-user-blk-pci in qemu and with nvme ssd(bdev), i cant'
>> >> have access to the nvme feature in guest(shadow door bell). is that
>> correct?
>> >
>> >Correct.
>> >
>> >> 2. in the vfio-user solution, does the interrupt sending from the
>> ssd(nvme)
>> >> go through the qemu/kvm? or it go straight to the guest kernel?
>> >
>> >It doesn't go to QEMU/KVM. It depends on how you've set it up in SPDK: it
>> can either go the host kernel or to SPDK.
>> >
>> >> 3. when will the vfio-user be available? does kvm have same delima here?
>> >
>> >vfio-user is under review in QEMU, so we can't predict when it will be
>> accepted upstream. This doesn't mean you can't use it though, have a look
>> here: https://urldefense.proofpoint.com/v2/url?u=https-
>> 3A__github.com_nutanix_libvfio-
>> 2Duser_blob_master_docs_spdk.md&d=DwIGaQ&c=s883GpUCOChKOHiocY
>> tGcg&r=XTpYsh5Ps2zJvtw6ogtti46atk736SI4vgsJiUKIyDE&m=5xzcHgA4AEwEk
>> wrWO3u9vZt5Fmg2QcvFmOxjMr4NL5Usz4o_2HIoCisBygQ9-
>> D1Q&s=k2t1I2xi9Och5KIeiaXyo2wo9wFbhIyALcMUph5xLK0&e=
>> >
>> >>
>> >>
>> >> thank you very much!
>> >>
>> >>
>> >>
>> >>
>> >> ---- 回复的原邮件 ----
>> >> | 发件人 | Liu, Changpeng<changpeng.liu(a)intel.com> |
>> >> | 日期 | 2022年03月08日 16:33 |
>> >> | 收件人 | Storage Performance Development Kit<spdk(a)lists.01.org> |
>> >> | 抄送至 | |
>> >> | 主题 | [SPDK] Re: The difference between vhost-nvme and vhost-blk |
>> >> Previously SPDK extended QEMU with a separate driver to enable vhost-
>> >> nvme, this driver
>> >> was not accepted by QEMU, so now we support emulated NVMe with a
>> new
>> >> solution
>> >> "vfio-user", again the driver for supporting this is still under code review
>> of
>> >> QEMU community,
>> >> but SPDK already supports this.
>> >>
>> >> > -----Original Message-----
>> >> > From: zbhhbz(a)yeah.net <zbhhbz(a)yeah.net>
>> >> > Sent: Tuesday, March 8, 2022 4:29 PM
>> >> > To: spdk(a)lists.01.org
>> >> > Subject: [SPDK] The difference between vhost-nvme and vhost-blk
>> >> >
>> >> > Could someone help me understand the difference between vhost-
>> nvme
>> >> and
>> >> > vhost-blk?
>> >> > The online doc only shows there is vhost-user-blk-pci, why not vhost-
>> user-
>> >> nvme-
>> >> > pci?
>> >> > There is little doc fined in github/spdk and the qemu itself doesn't help
>> >> either
>> >> > Thanks!
>> >> > _______________________________________________
>> >> > SPDK mailing list -- spdk(a)lists.01.org
>> >> > To unsubscribe send an email to spdk-leave(a)lists.01.org
>> >> _______________________________________________
>> >> SPDK mailing list -- spdk(a)lists.01.org
>> >> To unsubscribe send an email to spdk-leave(a)lists.01.org
>> >> _______________________________________________
>> >> SPDK mailing list -- spdk(a)lists.01.org
>> >> To unsubscribe send an email to spdk-leave(a)lists.01.org
>> >_______________________________________________
>> >SPDK mailing list -- spdk(a)lists.01.org
>> >To unsubscribe send an email to spdk-leave(a)lists.01.org
>> _______________________________________________
>> SPDK mailing list -- spdk(a)lists.01.org
>> To unsubscribe send an email to spdk-leave(a)lists.01.org
>_______________________________________________
>SPDK mailing list -- spdk(a)lists.01.org
>To unsubscribe send an email to spdk-leave(a)lists.01.org