[dpdk-users] dpdk with snort 2.9.9 receive from DPDK ring

Ming Fu Ming.Fu at esentire.com
Fri Sep 1 16:17:11 CEST 2017
Previous message: [dpdk-users] How rte_mempool_ops_table is populated in a DPDK client application?
Next message: [dpdk-users] dpdk with snort 2.9.9 receive from DPDK ring
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi Tom,

Thanks for share with me the dpdkring DAQ. I tried to use it with dpdk-17.08. It seems to be coded for an older version of DPDK, some API call has missing argument. I assume these are extras added to the newer version of DPDK.

Conceptually, my code is the very similar to yours. However, I haven't manage to include 
	include $(RTE_SDK)/mk/rte.vars.mk
	include $(RTE_SDK)/mk/rte.extlib.mk
into the daq Makefile for dpdkring. Do you know if you have to include any DPDK mk include file from DPDK mk directory? I didn't find any such change to the DAQ Makefile from your git repo, but the DPDK doc seems to suggest such.

Ming

-----Original Message-----
From: tom.barbette at ulg.ac.be [mailto:tom.barbette at ulg.ac.be] 
Sent: August-30-17 7:55 AM
To: Andriy Berestovskyy <aber at semihalf.com>
Cc: Padam Jeet Singh <padam.singh at inventum.net>; Ming Fu <Ming.Fu at esentire.com>; users at dpdk.org
Subject: Re: [dpdk-users] dpdk with snort 2.9.9 receive from DPDK ring

Hi all,

We have a proof of concept but working implementation of DPDK ring in snort available here : https://github.com/IurmanJ/daq-2.0.6/. It is a clone of the daq-2.0.6, patched for DPDK and DPDK ring support. 
The interesting file would be https://github.com/IurmanJ/daq-2.0.6/blob/master/os-daq-modules/daq_dpdkring.c . I guess you can use it to compare the difference, or use that one.

For autorship notice, it has been made by Justin Iurman for his master thesis.

Cheers,

Tom Barbette
PhD Student @ Université de Liège 

----- Mail original -----
> De: "Andriy Berestovskyy" <aber at semihalf.com>
> À: "Padam Jeet Singh" <padam.singh at inventum.net>
> Cc: "Ming Fu" <Ming.Fu at esentire.com>, users at dpdk.org
> Envoyé: Mercredi 30 Août 2017 11:18:39
> Objet: Re: [dpdk-users] dpdk with snort 2.9.9 receive from DPDK ring

> Hey,
> 1. There is a non-EAL thread support since DPDK 2.0, so any thread can 
> call rte_pktmbuf_free()
> 
> 2. From the traceback it looks like the memory pool enqueue() op is 
> NULL for a reason. Note that to free an mbuf, the mbuf header should 
> be valid and point to a valid memory pool mbuf belongs.
> 
> Try to compile DPDK in debug mode and you might see the reason.
> 
> Andriy
> 
> On Wed, Aug 30, 2017 at 7:15 AM, Padam Jeet Singh 
> <padam.singh at inventum.net> wrote:
>>
>>> On 30-Aug-2017, at 2:21 AM, Ming Fu <Ming.Fu at esentire.com> wrote:
>>>
>>> I am making a snort DAQ module to receive packet from DPDK capture 
>>> through a ring. The snort version is 2.9.9 and DPDK version is 
>>> 17.08. The code is very similar to the 
>>> multi_process/client_server_mp example. The DPDK is initialized in 
>>> daq initialize function and mbuf received from daq acquire function. 
>>> Snort inspects the mbuf but does not re-inject back to the network, 
>>> so daq free the mbuf by rte_pktmbuf_free(mbuf) in the same thread as 
>>> it calls rte_ring_dequeue_burst(); The snort successfully received 
>>> the first burst of 32 packets, but it fails on the first rte_pktmbuf_free().
>>>
>>> Seems some internal dpdk function pointer is NULL.
>>> #0  0x0000000000000000 in ?? ()
>>> #1  0x00000000004f7a8d in rte_mempool_ops_enqueue_bulk (n=<optimized 
>>> out>, obj_table=<optimized out>, mp=<optimized out>)
>>>    at /home/mfu/work/dpdk-17.08/build/include/rte_mempool.h:497
>>> #2  __mempool_generic_put (cache=<optimized out>, n=1, 
>>> obj_table=0x7fffffffe6e8, mp=<optimized out>) at
>>> /home/mfu/work/dpdk-17.08/build/include/rte_mempool.h:1069
>>> #3  rte_mempool_generic_put (flags=<optimized out>, cache=<optimized 
>>> out>, n=1, obj_table=0x7fffffffe6e8, mp=<optimized out>)
>>>    at /home/mfu/work/dpdk-17.08/build/include/rte_mempool.h:1109
>>> #4  rte_mempool_put_bulk (n=1, obj_table=0x7fffffffe6e8, 
>>> mp=<optimized out>) at
>>> /home/mfu/work/dpdk-17.08/build/include/rte_mempool.h:1132
>>> #5  rte_mempool_put (obj=<optimized out>, mp=<optimized out>) at
>>> /home/mfu/work/dpdk-17.08/build/include/rte_mempool.h:1150
>>> #6  rte_mbuf_raw_free (m=<optimized out>) at
>>> /home/mfu/work/dpdk-17.08/build/include/rte_mbuf.h:853
>>> #7  rte_pktmbuf_free_seg (m=<optimized out>) at
>>> /home/mfu/work/dpdk-17.08/build/include/rte_mbuf.h:1349
>>> #8  rte_pktmbuf_free (m=<optimized out>) at
>>> /home/mfu/work/dpdk-17.08/build/include/rte_mbuf.h:1369
>>> #9  dpdk_daq_acquire (handle=0x1af6910, cnt=0, callback=<optimized 
>>> out>, metaback=<optimized out>, user=<optimized out>) at 
>>> daq_dpdk.c:234
>>> #10 0x00000000004551d3 in DAQ_Acquire ()
>>> #11 0x0000000000437828 in SnortMain ()
>>> #12 0x00007ffff61af830 in __libc_start_main () from
>>> /lib/x86_64-linux-gnu/libc.so.6
>>> #13 0x0000000000406d29 in _start ()
>>>
>>> I notice that snort has three threads,
>>> (gdb) info thread
>>>  Id   Target Id         Frame
>>> * 1    Thread 0x7ffff7fdf8c0 (LWP 42472) "snort" 0x0000000000000000 in ?? ()
>>>  2    Thread 0x7fffdff5a700 (LWP 42481) "eal-intr-thread" 0x00007ffff62969d3 in
>>>  epoll_wait () from /lib/x86_64-linux-gnu/libc.so.6
>>>  3    Thread 0x7fffdf4d5700 (LWP 42482) "snort" 0x00007ffff625b30d in nanosleep
>>>  () from /lib/x86_64-linux-gnu/libc.so.6
>>>
>>> Would it be a problem if the rte_pktmbuf_free() is not called from 
>>> the eal thread?
>>>
>>
>> The rte_mbuf_free internally calls rte_mempool put calls - which 
>> should only be called from a EAL thread - so yes, only call 
>> rte_pktmbuf_free from an EAL thread.
>>
>>> Thanks
>>> Ming
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> ---------------------------------------------------------------------
>> -------------
>> NOTICE
>>
>> Please Consider the Environment before printing this Email.
>>
>> This email was sent from within Inventum Technologies Private Limited 
>> (https://www.inventum.net). This email (and any attachments or 
>> hyperlinks within it) may contain information that is confidential, 
>> legally privileged or otherwise protected from disclosure. If you are 
>> not the intended recipient of this email, you are not entitled to 
>> use, disclose, distribute, copy, print, disseminate or rely on this 
>> email in any way. If you have received this email in error, please 
>> notify the sender immediately by telephone or email and destroy it, and all copies of it.
>>
>> We have taken steps to ensure that this email (and any attachments) 
>> are free from computer viruses and the like. However, it is the 
>> recipient's responsibility to ensure that it is actually virus free. 
>> Any emails that you send to us may be monitored for the purposes of 
>> ascertaining whether the communication complies with the law and our policies.
>>
> 
> 
> 
> --
> Andriy Berestovskyy
Previous message: [dpdk-users] How rte_mempool_ops_table is populated in a DPDK client application?
Next message: [dpdk-users] dpdk with snort 2.9.9 receive from DPDK ring
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the users mailing list