See my reply to "mem: add API to obstain memory-backed file info" for a workaround. With fixes for that and the TUNSETVNETHDRSZ issue I was able to get traffic running over vhost-user.