[SPDK] thoughts of vhost-user disks + RDMA

Hi experts,

I have questions about getting data from the vhost-user and sending
out through RDMA with zero-copy.

When using vhost-user-blk/scsi, all VM's memory range is mapped to target.

For example, if a VM has a very big memory, like 100G, and it connects
to 10 vhost-user-blk disks.
Then the vhost-user target needs mmap 10*100G virtual memory.
If the data needs to transfer to another host through RDMA, and don't
want to copy the data out of vhost-user, you should register the all
virtual memory of vhost-user as RDMA MR at initialization. Because you
don't know the virtio data is put in which memory range, and only
register part of virtual memory.  right?
There will have so big range of memory that needs to be registered.
With some tests, the `ibv_reg_mr` call will make the host hang when
registering serval TB virtual memory.


If registering MR at the IO path, the performance is not acceptable.

Any ideas about this issue if don't want to copy data?

Thanks,

Feng Li

-- 
The SmartX email address is only for business purpose. Any sent message 
that is not related to the business is not authorized or permitted by 
SmartX.
本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.