[dpdk-dev] vhost-user technical isssues

[Tetsuya Mukawa]

Hi Xie,

(2014/11/12 6:37), Xie, Huawei wrote:
> Hi Tetsuya:
> There are two major technical issues in my mind for vhost-user implementation.
>
> 1) memory region map
> Vhost-user passes us file fd and offset for each memory region. Unfortunately the mmap offset is "very" wrong. I discovered this issue long time ago, and also found
> that I couldn't mmap the huge page file even with correct offset(need double check).
> Just now I find that people reported this issue on Nov 3.
> [Qemu-devel] [PULL 27/29] vhost-user: fix mmap offset calculation
> Anyway, I turned to the same idea used in our DPDK vhost-cuse: only use the fd for region(0) to map the  whole file.
> I think we should use this way temporarily to support qemu-2.1 as it has that bug.
I agree with you.
Also we may have an issue about un-mapping file on hugetlbfs of linux.
When I check munmap(), it seems 'size' need to be aligned by hugepage size.
(I guess it may be a kernel bug. Might be fixed already.)
Please add return value checking code for munmap().
Still munmap() might be failed.

>
> 2) what message is the indicator for vhost start/release?
> Previously  for vhost-cuse, it has SET_BACKEND message.
> What we should do for vhost-user?
> SET_VRING_KICK for start?
I think so.

> What about for release?
> Unlike the kernel virtio, the DPDK virtio in guest could be restarted. 
>
> Thoughts?
I guess we need to consider 2 types of restarting.
One is virtio-net driver restarting, the other is vhost-user backend
restarting.
But, so far, it's nice to start to think about virtio-net driver
restarting first.

Probably we need to implement a way to let vhost-user backend know
virtio-net driver is restarted.
I am not sure what is good way to let vhost-user backend know it.
But how about followings RFC?

- When unix domain socket is closed, vhost-user backend should treat it
as "release".
 It is useful when QEMU itself is gone suddenly.

- Also, implementing new ioctl command like VHOST_RESET_BACKEND.
 This command should be sent from virtio-net device of QEMU when
 VIRTIO_CONFIG_STATUS_RESET register of virtio-net device is set by
vrtio-net driver.
 (Usually this register is set when virtio-net driver is initialized or
stopped.)
 It means we need to change QEMU. ;)
 It seems virtio-net PMD already sets this register when PMD is
initialized or stopped.
 So this framework should work, and can let vhost-user backend know
driver resetting.
 (And I guess we can say same things for virtio-net kernel driver.)
 It might be enough to close an unix domain socket, instead of
implementing new command.
 But in the case, we may need auto reconnection mechanism.

- We also need to consider DPDK application is gone suddenly without
setting reset register.
 In the case, vhost-user backend cannot know it. Only user (or some kind
of watchdog
 applications on guest) knows it.
 Because of this, user(or app.) should have responsibility to solve this
situation.
 To be more precise, user should let vhost-user backend know device
releasing.
 If user starts an other DPDK application without solving the issue, the
new DPDK application may
 access memory that vhost-user backend is also accessing.
 I guess user can solve the issue using "dpdk_nic_bind.py".
 The script can move virtio-net device to kernel virtio-net driver, and
return it to igb_uio.
 While those steps, virtio-net device is initialized by virtio-net
kernel driver.
 So vhost-user backend can know device releasing.

Tetsuya

>
> -huawei