[dpdk-dev] [PATCH v3 2/2] vhost: Add VHOST PMD

[Wang, Zhihong]

> -----Original Message-----
> From: Tetsuya Mukawa [mailto:mukawa at igel.co.jp]
> Sent: Friday, November 13, 2015 11:10 AM
> To: Wang, Zhihong <zhihong.wang at intel.com>; dev at dpdk.org; Liu, Yuanhan
> <yuanhan.liu at intel.com>
> Cc: ann.zhuangyanying at huawei.com
> Subject: Re: [dpdk-dev] [PATCH v3 2/2] vhost: Add VHOST PMD
> 
> On 2015/11/12 21:52, Wang, Zhihong wrote:
> > Hi Tetsuya,
> >
> > In my test I created 2 vdev using "--vdev
> 'eth_vhost0,iface=/tmp/sock0,queues=1' --vdev
> 'eth_vhost1,iface=/tmp/sock1,queues=1'", and the qemu message got handled
> in wrong order.
> > The reason is that: 2 threads are created to handle message from 2 sockets, but
> their fds are SHARED, so each thread are reading from both sockets.
> >
> > This can lead to incorrect behaviors, in my case sometimes the
> VHOST_USER_SET_MEM_TABLE got handled after VRING initialization and lead to
> destroy_device().
> >
> > Detailed log as shown below: thread 69351 & 69352 are both reading fd 25.
> Thanks Yuanhan for helping debugging!
> >
> 
> Hi Zhihong and Yuanhan,
> 
> Thank you so much for debugging the issue.
> I will fix vhost PMD not to create multiple message handling threads.
> 
> I am going to submit the PMD today.
> Could you please check it again using latest one?
> 

Looking forward to it!


> Tetsuya
> 
> 
> > Thanks
> > Zhihong
> >
> >
> > ----------------------------------------------------------------------
> > -------------------------------------------
> >
> > ---->  debug: setting up new vq conn for fd: 23, tid: 69352
> > VHOST_CONFIG: new virtio connection is 25
> > VHOST_CONFIG: new device, handle is 0
> > ---->  debug: vserver_message_handler thread id: 69352, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_OWNER
> > ---->  debug: vserver_message_handler thread id: 69352, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_GET_FEATURES
> > ---->  debug: vserver_message_handler thread id: 69352, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
> > VHOST_CONFIG: vring call idx:0 file:26
> > ---->  debug: vserver_message_handler thread id: 69352, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
> > VHOST_CONFIG: vring call idx:1 file:27
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
> > VHOST_CONFIG: vring call idx:0 file:28
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL
> > VHOST_CONFIG: vring call idx:1 file:26
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_FEATURES
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_MEM_TABLE
> > ---->  debug: device_fh: 0: user_set_mem_table
> > VHOST_CONFIG: mapped region 0 fd:27 to 0x7ff6c0000000 sz:0xa0000
> > off:0x0
> > VHOST_CONFIG: mapped region 1 fd:29 to 0x7ff680000000 sz:0x40000000
> > off:0xc0000
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
> > VHOST_CONFIG: vring kick idx:0 file:30
> > ---->  debug: vserver_message_handler thread id: 69352, fd: 25
> > VHOST_CONFIG: virtio is not ready for processing.
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_BASE
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_ADDR
> > ---->  debug: vserver_message_handler thread id: 69351, fd: 25
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_KICK
> > VHOST_CONFIG: vring kick idx:1 file:31
> > VHOST_CONFIG: virtio is now ready for processing.
> > PMD: New connection established
> > VHOST_CONFIG: read message VHOST_USER_SET_VRING_NUM
> >
> > ----------------------------------------------------------------------
> > -------------------------------------------
> >
> >> ...
> >> +
> >> +static void *vhost_driver_session(void *param __rte_unused) {
> >> +	static struct virtio_net_device_ops *vhost_ops;
> >> +
> >> +	vhost_ops = rte_zmalloc(NULL, sizeof(*vhost_ops), 0);
> >> +	if (vhost_ops == NULL)
> >> +		rte_panic("Can't allocate memory\n");
> >> +
> >> +	/* set vhost arguments */
> >> +	vhost_ops->new_device = new_device;
> >> +	vhost_ops->destroy_device = destroy_device;
> >> +	if (rte_vhost_driver_pmd_callback_register(vhost_ops) < 0)
> >> +		rte_panic("Can't register callbacks\n");
> >> +
> >> +	/* start event handling */
> >> +	rte_vhost_driver_session_start();
> >> +
> >> +	rte_free(vhost_ops);
> >> +	pthread_exit(0);
> >> +}
> >> +
> >> +static void vhost_driver_session_start(struct pmd_internal
> >> +*internal) {
> >> +	int ret;
> >> +
> >> +	ret = pthread_create(&internal->session_th,
> >> +			NULL, vhost_driver_session, NULL);
> >> +	if (ret)
> >> +		rte_panic("Can't create a thread\n"); }
> >> +
> >> ...