[dpdk-dev] mutli process C/S model example init failed on xen dom0 with dpdk-16.07 rc2 package

[Sergio Gonzalez Monroy]

Hi,

On 12/07/2016 12:30, Olivier MATZ wrote:
> Hi Huilong,
>
>
> On 07/12/2016 11:22 AM, Xu, HuilongX wrote:
>> Hi all,
>>
>> I run mutli procee C/S model example failed on xen dom0. Does anyone
>> give me some suggest how to debug it?
>>
>> Thanks a lot
>>
>> test environment:
>>
>>        OS&kernel: 3.17.4-301.fc21.x86_64
>>
>> Gcc version: gcc version 4.9.2 20141101 (Red Hat 4.9.2-1) (GCC)
>>
>> Package :dpdk.16.07-rc1.tar.gz
>>
>> Target: x86_64-native-linuxapp-gcc
>>
>> Compile switch: enable CONFIG_RTE_LIBRTE_XEN_DOM0
>>
>> Xen version:4.4.1
>>
>> Test cmdline and result:
>>
>> /examples/multi_process/client_server_mp/mp_server/mp_server/x86_64-native-linuxapp-gcc/mp_server 
>>
>> -c f -n 4 --xen-dom0 -- -p 0x3 -n 2
>> EAL: Detected 72 lcore(s)
>> EAL: Probing VFIO support...
>> PMD: bnxt_rte_pmd_init() called for (null)
>> EAL: PCI device 0000:01:00.0 on NUMA socket 0
>> EAL: probe driver: 8086:1521 rte_igb_pmd
>> EAL: PCI device 0000:01:00.1 on NUMA socket 0
>> EAL: probe driver: 8086:1521 rte_igb_pmd
>> EAL: PCI device 0000:04:00.0 on NUMA socket 0
>> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
>> EAL: PCI device 0000:04:00.1 on NUMA socket 0
>> EAL: probe driver: 8086:10fb rte_ixgbe_pmd
>> Creating mbuf pool 'MProc_pktmbuf_pool' [6144 mbufs] ...
>> Port 0 init ... Segmentation fault (core dumped)
>>
>
> I reproduced the issue on my platform. In my case, the crash occurs in 
> rx_queue_setup():
>
>         /* Free memory prior to re-allocation if needed. */
>         if (dev->data->rx_queues[queue_idx] != NULL) {
> => em_rx_queue_release(dev->data->rx_queues[queue_idx]);
>                 dev->data->rx_queues[queue_idx] = NULL;
>         }
>
> I don't this we should go in that area for the first rx queue 
> initialization. I suspect it could be related to this commit:
> http://dpdk.org/browse/dpdk/commit/?id=ea0bddbd14e68f
>
> I think we cannot expect that memory is initialized at 0 when using 
> Xen dom0. If I add the following (dirty) patch, I don't see a crash 
> anymore:

I don't have a Xen system available right now, but I'm not sure I follow 
here.
Are you saying that when we allocate pages/hugepages from Xen they are 
not zeroed?

>
> --- a/lib/librte_eal/common/eal_common_memzone.c
> +++ b/lib/librte_eal/common/eal_common_memzone.c
> @@ -258,6 +258,8 @@ memzone_reserve_aligned_thread_unsafe(const char 
> *name, size_t len,
>         mz->flags = 0;
>         mz->memseg_id = elem->ms - 
> rte_eal_get_configuration()->mem_config->memseg;
>
> +       memset(mz->addr, 0, mz->len);
> +
>         return mz;
>  }
>

The commit you are referring to does not touch the memzone reserve APIs, 
only changes zmalloc and related APIs.

> --- a/lib/librte_eal/common/rte_malloc.c
> +++ b/lib/librte_eal/common/rte_malloc.c
> @@ -123,7 +123,13 @@ rte_malloc(const char *type, size_t size, 
> unsigned align)
>  void *
>  rte_zmalloc_socket(const char *type, size_t size, unsigned align, int 
> socket)
>  {
> -       return rte_malloc_socket(type, size, align, socket);
> +       void *x = rte_malloc_socket(type, size, align, socket);
> +
> +       if (x == NULL)
> +               return NULL;
> +
> +       memset(x, 0, size);
> +       return x;
>  }
>
>  /*
>
>
> Sergio, could you have a look at it?
>
> Regards,
> Olivier