[dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci

Zhang, Helin helin.zhang at intel.com
Sun Jun 5 17:47:15 CEST 2016
Previous message: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
Next message: [dpdk-dev] [PATCH 0/2] NSH packet type support in i40e
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi Kannan

Could you help to provide more details of your test cases? How to reproduce that issue? It would be better to try with DPDK example applications.
I think the kernel driver version, DPDK version, XL710 firmware version are the basic information we want to know.
If possible, I'd suggest you to try with latest version of above and try if the issue is still there.
Thank you very much!

Regards,
Helin

> -----Original Message-----
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Chandrasekar
> Kannan
> Sent: Thursday, June 2, 2016 2:22 AM
> To: dev at dpdk.org
> Subject: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
> 
> I have been attempting to VPP (dpdk) application with SR-IOV enabled.
> Using the vfio-pci driver/i40e/XL710nics/.
> I'm encountering a SEGV on the i40e code path.
> 
> Has anyone else seen this behaviour ?.
> 
> full mail thread discussion with vpp-dev is here -
> https://lists.fd.io/pipermail/vpp-dev/2016-June/001348.html
> 
> 
> --------------------------------
> here's a backtrace during vpp crash...
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 0x7f2b811fd700 (LWP 8725)]
> 0x0000000000455539 in rx_recv_pkts ()
> (gdb) backtrace
> #0  0x0000000000455539 in rx_recv_pkts ()
> #1  0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2  0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3  dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> xd=0x7f2bb1898540)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> #4  dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5  dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc,
> f=<optimized
> out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6  0x00007f2be47eb77a in dispatch_node (vm=vm at entry=0x7f2bb1f02cd4,
> node=node at entry=0x7f2bb1ef59fc,
> type=type at entry=VLIB_NODE_TYPE_INPUT,
>     dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING,
> frame=frame at entry=0x0, last_time_stamp=1482097661494035)
>     at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7  0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8  dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b811fcc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop () ---Type <return> to continue,
> or q <return> to quit---
> #12 0x0000000000000000 in ?? ()
> (gdb)
> (gdb) thread apply all bt
> 
> Thread 10 (Thread 0x7f2b8c685700 (LWP 8722)):
> #0  0x00007f2be32584ad in accept () at ../sysdeps/unix/syscall-template.S:81
> #1  0x00000000004ef178 in pci_vfio_mp_sync_thread ()
> #2  0x00007f2be3251dc5 in start_thread (arg=0x7f2b8c685700) at
> pthread_create.c:308
> #3  0x00007f2be2992ced in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
> 
> Thread 9 (Thread 0x7f2b821ff700 (LWP 8723)):
> #0  0x00007f2be29932c3 in epoll_wait () at
> ../sysdeps/unix/syscall-template.S:81
> #1  0x00000000004efe04 in eal_intr_thread_main ()
> #2  0x00007f2be3251dc5 in start_thread (arg=0x7f2b821ff700) at
> pthread_create.c:308
> #3  0x00007f2be2992ced in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:113
> 
> Thread 8 (Thread 0x7f2b819fe700 (LWP 8724)):
> #0  0x000000000045526b in rx_recv_pkts ()
> #1  0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2  0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1e75580, queue_id=0, port_id=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3  dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
> xd=0x7f2bb1a1fd40)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> ---Type <return> to continue, or q <return> to quit---
> #4  dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=0,
> cpu_index=<optimized out>, node=0x7f2bb1ea18cc, xd=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5  dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ea18cc,
> f=<optimized
> out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6  0x00007f2be47eb77a in dispatch_node (vm=vm at entry=0x7f2bb1f237d4,
> node=node at entry=0x7f2bb1ea18cc,
> type=type at entry=VLIB_NODE_TYPE_INPUT,
>     dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING,
> frame=frame at entry=0x0, last_time_stamp=1482097666654800)
>     at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7  0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1f237d4)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8  dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b819fdc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop ()
> #12 0x0000000000000000 in ?? ()
> 
> Thread 7 (Thread 0x7f2b811fd700 (LWP 8725)):
> #0  0x0000000000455539 in rx_recv_pkts ()
> #1  0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2  0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1e66a80, queue_id=1, port_id=<optimized out>) ---Type
> <return> to continue, or q <return> to quit---
>     at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3  dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> xd=0x7f2bb1898540)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> #4  dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=1,
> cpu_index=<optimized out>, node=0x7f2bb1ef59fc, xd=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5  dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ef59fc,
> f=<optimized
> out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6  0x00007f2be47eb77a in dispatch_node (vm=vm at entry=0x7f2bb1f02cd4,
> node=node at entry=0x7f2bb1ef59fc,
> type=type at entry=VLIB_NODE_TYPE_INPUT,
>     dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING,
> frame=frame at entry=0x0, last_time_stamp=1482097661494035)
>     at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7  0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1f02cd4)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8  dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b811fcc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop ()
> #12 0x0000000000000000 in ?? ()
> 
> Thread 6 (Thread 0x7f2b809fc700 (LWP 8726)):
> ---Type <return> to continue, or q <return> to quit---
> #0  0x000000000045539e in rx_recv_pkts ()
> #1  0x0000000000455e76 in i40e_recv_pkts_bulk_alloc ()
> #2  0x00007f2be43d8bf7 in rte_eth_rx_burst (nb_pkts=256,
> rx_pkts=0x7f2bb1da8a00, queue_id=2, port_id=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-root/install-vpp-
> native/dpdk/include/rte_ethdev.h:2641
> #3  dpdk_rx_burst (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
> xd=0x7f2bb1898540)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/dpdk_priv.h:64
> #4  dpdk_device_input (dm=0x7f2be4673980 <dpdk_main>, queue_id=2,
> cpu_index=<optimized out>, node=0x7f2bb1ebc7f8, xd=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:508
> #5  dpdk_input_rss (vm=<optimized out>, node=0x7f2bb1ebc7f8,
> f=<optimized
> out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/node.c:821
> #6  0x00007f2be47eb77a in dispatch_node (vm=vm at entry=0x7f2bb1ee29d4,
> node=node at entry=0x7f2bb1ebc7f8,
> type=type at entry=VLIB_NODE_TYPE_INPUT,
>     dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING,
> frame=frame at entry=0x0, last_time_stamp=1482097658614609)
>     at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #7  0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1ee29d4)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #8  dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #9  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #10 0x00007f2b809fbc40 in ?? ()
> #11 0x00000000004eb9c7 in eal_thread_loop () ---Type <return> to continue,
> or q <return> to quit---
> #12 0x0000000000000000 in ?? ()
> 
> Thread 5 (Thread 0x7f2b801fb700 (LWP 8727)):
> #0  0x00007f2be47eb728 in dispatch_node (vm=vm at entry=0x7f2bb1ec2614,
> node=node at entry=0x7f2bb1da5e60,
> type=type at entry=VLIB_NODE_TYPE_INPUT,
>     dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING,
> frame=frame at entry=0x0, last_time_stamp=1482097666665442)
>     at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:924
> #1  0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1ec2614)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #2  dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #3  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #4  0x00007f2b801fac40 in ?? ()
> #5  0x00000000004eb9c7 in eal_thread_loop ()
> #6  0x0000000000000000 in ?? ()
> 
> Thread 4 (Thread 0x7f2b7f9fa700 (LWP 8728)):
> #0  0x00007f2be43db6cc in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1ea2294)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #1  dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> ---Type <return> to continue, or q <return> to quit---
> #2  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #3  0x00007f2b7f9f9c40 in ?? ()
> #4  0x00000000004eb9c7 in eal_thread_loop ()
> #5  0x0000000000000000 in ?? ()
> 
> Thread 3 (Thread 0x7f2b7f1f9700 (LWP 8729)):
> #0  vlib_node_runtime_update_stats (n_clocks=205, n_vectors=0, n_calls=1,
> node=0x7f2bb1d782d0, vm=0x7f2bb1e81f94)
>     at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:629
> #1  dispatch_node (vm=vm at entry=0x7f2bb1e81f94,
> node=node at entry=0x7f2bb1d782d0,
> type=type at entry=VLIB_NODE_TYPE_INPUT,
>     dispatch_state=dispatch_state at entry=VLIB_NODE_STATE_POLLING,
> frame=frame at entry=0x0, last_time_stamp=1482097666680456)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/main.c:1005
> #2  0x00007f2be43db6d8 in dpdk_worker_thread_internal
> (have_io_threads=0, callback=0x0, vm=0x7f2bb1e81f94)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:209
> #3  dpdk_worker_thread (w=<optimized out>, io_name=<optimized out>,
> callback=0x0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vnet/vnet/devices/dpdk/threads.c:265
> #4  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #5  0x00007f2b7f1f8c40 in ?? ()
> #6  0x00000000004eb9c7 in eal_thread_loop ()
> #7  0x0000000000000000 in ?? ()
> 
> ---Type <return> to continue, or q <return> to quit--- Thread 2 (Thread
> 0x7f2b7e9f8700 (LWP 8730)):
> #0  0x00007f2be325896d in nanosleep () at
> ../sysdeps/unix/syscall-template.S:81
> #1  0x0000000000413916 in ip46_fib_stats_delay.isra.6 ()
> #2  0x00000000004190b6 in stats_thread_fn ()
> #3  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #4  0x00007f2b7e9f7da0 in ?? ()
> #5  0x00007f2be3251dc5 in start_thread (arg=0x0) at pthread_create.c:308
> #6  0x0000000000000000 in ?? ()
> 
> Thread 1 (Thread 0x7f2be52898c0 (LWP 8721)):
> #0  0x00007f2be2992f27 in __GI_epoll_pwait (epfd=5,
> events=0x7f2bb2155424, maxevents=256, timeout=timeout at entry=10,
>     set=set at entry=0x7f2be4c522e0 <unblock_all_signals.16333>) at
> ../sysdeps/unix/sysv/linux/epoll_pwait.c:48
> #1  0x00007f2be4a429fb in linux_epoll_input (vm=0x7e13c0
> <vlib_global_main>, node=<optimized out>, frame=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/unix/input.c:146
> #2  0x00007f2be47eb77a in dispatch_node (vm=0x7e13c0
> <vlib_global_main>, node=0x7f2bb2254cc0, type=<optimized out>,
> dispatch_state=<optimized out>,
>     frame=<optimized out>, last_time_stamp=1482097636972836) at
> /w/workspace/vpp-merge-master-centos7/build-data/../vlib/vlib/main.c:996
> #3  0x00007f2be47ebea1 in vlib_main_loop (vm=0x7e13c0
> <vlib_global_main>) at /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/main.c:1403
> #4  vlib_main (vm=vm at entry=0x7e13c0 <vlib_global_main>,
> input=input at entry
> =0x7f2bb2786fa0)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/main.c:1615
> #5  0x00007f2be4a435f3 in thread0 (arg=8262592) at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/unix/main.c:425
> ---Type <return> to continue, or q <return> to quit---
> #6  0x00007f2be3791530 in clib_calljmp () at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vppinfra/vppinfra/longjmp.S:110
> #7  0x00007ffc98a68d60 in ?? ()
> #8  0x00007f2be4a43e31 in vlib_unix_main (argc=<optimized out>,
> argv=<optimized out>)
>     at
> /w/workspace/vpp-merge-master-centos7/build-
> data/../vlib/vlib/unix/main.c:485
> #9  0x0000000000000000 in ?? ()
> (gdb)
> (gdb)
> (gdb)
> (gdb)
> -------------------------------------------------------------------
Previous message: [dpdk-dev] SR-IOV/DPDK/VPP with vfio-pci
Next message: [dpdk-dev] [PATCH 0/2] NSH packet type support in i40e
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the dev mailing list