[dpdk-users] segmentation fault: malloc_elem_alloc()-> LIST_REMOVE() [FreeBSD 10.3, dpdk-16.04)

[Sergio Gonzalez Monroy]

Hi,

Not sure if it is a new feature in FreeBSD but do you have ASLR enabled?

Sergio

On 01/08/2016 19:02, txcy uio wrote:
> Hello all,
>
> While running as a secondary process I received the segmentation fault as
> below on FreeBSD 10.3 and dpdk 16.04:
>
> Program received signal SIGSEGV, Segmentation fault -
>
> [Switching to Thread 801c06400 (LWP 100363/test_client)]
> 0x0000000000416dcf in malloc_elem_alloc (elem=0x838e57000, size=64,
> align=<value optimized out>, bound=<value optimized out>)
>      at dpdk-16.04/lib/librte_eal/common/malloc_elem.c:196
> 196 LIST_REMOVE(elem, free_list);
> Current language:  auto; currently minimal
> (gdb) bt
> #0  0x0000000000416dcf in malloc_elem_alloc (elem=0x838e57000, size=64,
> align=<value optimized out>,
>      bound=<value optimized out>) at
> dpdk-16.04/lib/librte_eal/common/malloc_elem.c:196
> #1  0x00000000004174a7 in malloc_heap_alloc (heap=0x800698a1c, type=<value
> optimized out>, size=64,
>      flags=<value optimized out>, align=<value optimized out>, bound=0)
>      at dpdk-16.04/lib/librte_eal/common/malloc_heap.c:168
> #2  0x0000000000416416 in rte_malloc_socket (type=0x41fdf8
> "RING_TAILQ_ENTRY", size=24, align=<value optimized out>,
>      socket_arg=Error accessing memory address 0xffffffffffffffff: Bad
> address.
> ) at dpdk-16.04/lib/librte_eal/common/rte_malloc.c:91
> #3  0x00000000004164d4 in rte_zmalloc (type=0x838e57000 "\034jj", size=24,
> align=0)
>      at dpdk-16.04/lib/librte_eal/common/rte_malloc.c:126
> #4  0x0000000000419044 in rte_ring_create (name=0x7fffffffe7e0
> "spdk_active_pollers_6", count=4096, socket_id=0, flags=3)
>      at dpdk-16.04/lib/librte_ring/rte_ring.c:177
>
> Segmentation fault happened in line 518 below (sys/queue.h)
>
> 510 #define LIST_REMOVE(elm, field) do {
>   \
> 511         QMD_SAVELINK(oldnext, (elm)->field.le_next);
>   \
> 512         QMD_SAVELINK(oldprev, (elm)->field.le_prev);
>   \
> 513         QMD_LIST_CHECK_NEXT(elm, field);
>   \
> 514         QMD_LIST_CHECK_PREV(elm, field);
>   \
> 515         if (LIST_NEXT((elm), field) != NULL)
>   \
> 516                 LIST_NEXT((elm), field)->field.le_prev =
>   \
> 517                     (elm)->field.le_prev;
> \
> 518         *(elm)->field.le_prev = LIST_NEXT((elm), field);
>   \
> 519         TRASHIT(*oldnext);
>   \
> 520         TRASHIT(*oldprev);
>   \
> 521 } while (0)
> 522
>
> It seems like the le_prev is not a valid address when being accessed from
> the secondary process however the same address is valid when accessed from
> the primary process (see below). Any idea what is going on here ?
>
>
> Secondary process :
>
> (gdb) p rte_config
> $1 = {master_lcore = 7, lcore_count = 2, lcore_role = {ROLE_OFF, ROLE_OFF,
> ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_OFF, ROLE_RTE,
>      ROLE_RTE, ROLE_OFF <repeats 120 times>}, process_type =
> RTE_PROC_SECONDARY, mem_config = 0x800666000}
>
> (gdb) p (struct rte_mem_config)*0x800666000
> $2 = {magic = 19820526, nchannel = 4, nrank = 0, mlock = {cnt = 0}, qlock =
> {cnt = 0}, mplock = {cnt = 0}, memzone_cnt = 22,
>    memseg = {{phys_addr = 6442450944, {addr = 0x802000000, addr_64 =
> 34393292800}, len = 1073741824,
>        hugepage_sz = 1073741824, socket_id = 0, nchannel = 0, nrank = 0},
> {phys_addr = 0, {addr = 0x0, addr_64 = 0}, len = 0,
>        hugepage_sz = 0, socket_id = 0, nchannel = 0, nrank = 0} <repeats 255
> times>}, memzone = {{
> .....
> malloc_heaps = {{lock = {locked = 1},
>        free_head = {{lh_first = 0x0}, {lh_first = 0x0}, {lh_first =
> 0x838e57000}, {lh_first = 0x0}, {lh_first = 0x0}, {
>            lh_first = 0x0}, {lh_first = 0x0}, {lh_first = 0x0}, {lh_first =
> 0x0}, {lh_first = 0x0}, {lh_first = 0x0},
> ..
>
> (gdb) p (struct malloc_elem)*0x838e57000
> $3 = {heap = 0x8006a6a1c, prev = 0x838dd6fc0, free_list = {le_next =
> 0x838e59000, le_prev = 0x8006a6a34}, ms = 0x80067401c,
>    state = ELEM_FREE, pad = 0, size = 4032}
>
> (gdb) p (struct malloc_elem)*0x838e59000
> $4 = {heap = 0x8006a6a1c, prev = 0x838e57fc0, free_list = {le_next =
> 0x838e5e000, le_prev = 0x8006a6a34}, ms = 0x80067401c,
>    state = ELEM_FREE, pad = 0, size = 4032}
>
> *(gdb) p (struct malloc_elem)*0x8006a6a34*
> *Error accessing memory address 0x8006a6a34: Bad address.*
>
> Primary process:
>
> (gdb) p rte_config
> $1 = {master_lcore = 0, lcore_count = 6, lcore_role = {ROLE_RTE, ROLE_RTE,
> ROLE_RTE, ROLE_RTE, ROLE_RTE, ROLE_RTE,
>      ROLE_OFF <repeats 122 times>}, process_type = RTE_PROC_PRIMARY,
> mem_config = 0x800674000}
>
> (gdb) p (struct rte_mem_config)*0x800674000
> $2 = {magic = 19820526, nchannel = 4, nrank = 0, mlock = {cnt = 0}, qlock =
> {cnt = 0}, mplock = {cnt = 0}, memzone_cnt = 22,
>    memseg = {{phys_addr = 6442450944, {addr = 0x802000000, addr_64 =
> 34393292800}, len = 1073741824,
>        hugepage_sz = 1073741824, socket_id = 0, nchannel = 0, nrank = 0},
> {phys_addr = 0, {addr = 0x0, addr_64 = 0}, len = 0,
> ....
> malloc_heaps = {{lock = {locked = 1},
>        free_head = {{lh_first = 0x0}, {lh_first = 0x0}, {lh_first =
> 0x838e57000},
> ...
>
>
> (gdb) p (struct malloc_elem)*0x838e57000
> $3 = {heap = 0x8006a6a1c, prev = 0x838dd6fc0, free_list = {le_next =
> 0x838e59000, le_prev = 0x8006a6a34}, ms = 0x80067401c,
>    state = ELEM_FREE, pad = 0, size = 4032}
>
> *(gdb) p (struct malloc_elem)*0x8006a6a34*
> *$4 = {heap = 0x838e57000, prev = 0x0, free_list = {le_next = 0x0, le_prev
> = 0x0}, ms = 0x0, state = ELEM_FREE, pad = 0,*
> *  size = 0}*
>
>
> --Tyc