[dpdk-users] pdump help

Pattan, Reshma reshma.pattan at intel.com
Mon Jul 25 22:11:10 CEST 2016
Previous message: [dpdk-users] pdump help
Next message: [dpdk-users] Possible rte_ring issues
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi Nishant,

Comments inline

Thanks,
Reshma
From: Nishant Verma [mailto:vnish11 at gmail.com]
Sent: Monday, July 25, 2016 8:31 PM
To: Pattan, Reshma <reshma.pattan at intel.com>
Cc: users at dpdk.org
Subject: Re: pdump help

Hi Reshma,
1. Not sure what exactly is the issue behind this. But as of now i did 2 things, in place of running setup.sh, i tried to give make command, with DESTDIR.
Now i got exe.
So if i execute the dpdk_pdump from this folder(destination directory that i gave at the time of building DPDK), it is working fine.
But if i go in ./dpdk/app/pdump, build app here and then try to run it is not working.
Ok, I will check this issue and fix if it is a bug. That is good that you could run from DESTDIR, I always ran from DESTDIR.
2. Even their is no  packet to capture, it showing error "insufficient data_len of mbuf"
    this is how i run and end pdump execution.
"
The mbuf-size that you pass to dpdk_pdump command should be same or greater than the mbuf size that was used for mempool creation for the Rx packets inside the primary application.
Ex: in L3fwd, the size used is RTE_MBUF_DEFAULT_BUF_SIZE.
rte_pktmbuf_pool_create(s, nb_mbuf, MEMPOOL_CACHE_SIZE, 0, RTE_MBUF_DEFAULT_BUF_SIZE, socketid);

If you know that value you can pass it on, else I suggest don’t pass any value, then dpdk_pdump will use default mbuf size internally i.e. (RTE_MBUF_DEFAULT_BUF_SIZE).
Here you are using mbuf-size 20 which is far less than the what l3fwd is using, hence pdump library will throw this error in such cases.

Try suggested points and let me know if you see any more issues.

root at ubuntu:/home/oneweb/dpdk/install/bin# ./dpdk_pdump --file-prefix l3f -- --pdump 'port=0,queue=*,rx-dev=/tmp/rx-file.pcap,tx-dev=/tmp/tx-file.pcap,ring-size=1024,mbuf-size=20,total-num-mbufs=1025'
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the kernel.
EAL: This may cause issues with mapping memory into secondary processes
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL: probe driver: 8086:15ad rte_ixgbe_pmd
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL: probe driver: 8086:15ad rte_ixgbe_pmd
PMD: eth_ixgbe_dev_init(): No TX queues configured yet. Using default TX function.
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
EAL: PCI device 0000:05:00.1 on NUMA socket 0
EAL: probe driver: 8086:1521 rte_igb_pmd
PMD: Initializing pmd_pcap for eth_pcap_rx_0
PMD: Creating pcap-backed ethdev on numa socket 0
Port 2 MAC: 00 00 00 01 02 03
PMD: Initializing pmd_pcap for eth_pcap_tx_0
PMD: Creating pcap-backed ethdev on numa socket 0
Port 3 MAC: 00 00 00 01 02 03
^C

Signal 2 received, preparing to exit...
##### PDUMP DEBUG STATS #####
-packets dequeued: 0
-packets transmitted to vdev: 0
-packets freed: 0
"
Here is what i get on l3fwd side..
"
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf
PDUMP: User mempool: insufficient data_len of mbuf

"

Atlast it seems to be working, thanks for the much needed help.
Rgds,
Nishant


On Mon, Jul 25, 2016 at 12:21 PM, Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>> wrote:
Hi Nishant,

The possible reasons for below error are


1)      Either  below falg is set to “n” in config/common_base file, please set it to “y” and compile again.

If this falg is not in the config/comman_base for you, just check other config files where this flag is there and make necessary change.

# Compile software PMD backed by PCAP files

#

CONFIG_RTE_LIBRTE_PMD_PCAP=n



2)      (Or) you might be having dpdk code which don’t have below fix. If you don’t have this fix, please add this fix , compile and check again.

http://dpdk.org/dev/patchwork/patch/14709/

diff --git a/drivers/net/pcap/rte_eth_pcap.c b/drivers/net/pcap/rte_eth_pcap.c

index b341f93..7125823 100644

--- a/drivers/net/pcap/rte_eth_pcap.c

+++ b/drivers/net/pcap/rte_eth_pcap.c

@@ -1090,6 +1090,6 @@  static struct rte_driver pmd_pcap_drv = {

        .uninit = rte_pmd_pcap_devuninit,

 };



-PMD_REGISTER_DRIVER(pmd_pcap_drv, pcap);

-DRIVER_REGISTER_PARAM_STRING(pcap, "rx_pcap=<string> tx_pcap=<string> "

+PMD_REGISTER_DRIVER(pmd_pcap_drv, eth_pcap);

+DRIVER_REGISTER_PARAM_STRING(eth_pcap, "rx_pcap=<string> tx_pcap=<string> "


Please check these 2 items and make the necessary changes and let me know how do you proceed.

Thanks,
Reshma

From: Pattan, Reshma
Sent: Monday, July 25, 2016 5:01 PM
To: 'Nishant Verma' <vnish11 at gmail.com<mailto:vnish11 at gmail.com>>
Cc: users at dpdk.org<mailto:users at dpdk.org>
Subject: RE: pdump help

Hi,

Can you paste me file config/common_base content output

Thanks,
Reshma

From: Nishant Verma [mailto:vnish11 at gmail.com]
Sent: Monday, July 25, 2016 4:58 PM

To: Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>>
Cc: users at dpdk.org<mailto:users at dpdk.org>
Subject: Re: pdump help

Hi Reshma,
Error cause is still same, but i think their is some progress what i see in error log as of now.
Don't know what exact ican share that will help you in debug the issue, so as of now sharing HugePage output, l3fwd log & pdump error log.
If you need anything else, please let me know.

Option: 29

AnonHugePages:      2048 kB
HugePages_Total:    8192
HugePages_Free:     7680
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB




--

root at ubuntu:/home/oneweb/dpdk/examples/l3fwd/build/app# ./l3fwd -c 606 -n 4 --proc-type=primary  --socket-mem=1024 --file-prefix l3f -- -p 0x1 --config="(0,0,1),(0,1,2)"
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL: PCI device 0000:05:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
L3FWD: LPM or EM none selected, default LPM on
Initializing port 0 ... Creating queues: nb_rxq=2 nb_txq=4...  Address:0C:C4:7A:73:EF:14, Destination:02:00:00:00:00:00, Allocated mbuf pool on socket 0
LPM: Adding route 0x01010100 / 24 (0)
LPM: Adding route IPV6 / 48 (0)
===============================Initialized PDUMP
txq=1,0,0 txq=2,1,0 txq=9,2,0 txq=10,3,0

Skipping disabled port 1

Initializing rx queues on lcore 1 ... rxq=0,0,0
Initializing rx queues on lcore 2 ... rxq=0,1,0
Initializing rx queues on lcore 9 ...
Initializing rx queues on lcore 10 ...


Checking link status..................................................done
Port 0 Link Up - speed 10000 Mbps - full-duplex
L3FWD: entering main loop on lcore 2
L3FWD:  -- lcoreid=2 portid=0 rxqueueid=1
L3FWD: lcore 9 has nothing to do
L3FWD: entering main loop on lcore 1
L3FWD:  -- lcoreid=1 portid=0 rxqueueid=0
L3FWD: lcore 10 has nothing to do

-------



root at ubuntu:/home/oneweb/dpdk/app/pdump# ./dpdk_pdump --file-prefix l3f  -- --pdump 'port=0,queue=*,rx-dev=/tmp/rx-file.pcap,tx-dev=/tmp/tx-file.pcap,ring-size=1024,mbuf-size=20,total-num-mbufs=1025'
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the kernel.
EAL:    This may cause issues with mapping memory into secondary processes
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
PMD: eth_ixgbe_dev_init(): No TX queues configured yet. Using default TX function.
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL: PCI device 0000:05:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL: no driver found for eth_pcap_rx_0
EAL: Driver, cannot attach the device
EAL: Error - exiting with code: 1
  Cause: vdev creation failed:create_mp_ring_vdev:669
rgds,
Nishant

On Mon, Jul 25, 2016 at 4:57 AM, Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>> wrote:
Hi Nishant,

1)Since you are using  --file-prefix l3f in l3fwd command, you need to use same file prefix in dpdk_pdump command also. This solves below reported issue.

2)And also make sure l3fwd application initializes packet capture framework by calling rte_pdump_init () api from its main().
 Also should  call uninitialize  api rte_pdump_uninit() before exiting l3fwd application. Please refer app/test-pmd/testpmd.c file to see how these apis are used.

Ex:
int
main(int argc, char** argv)
{
»·······int  diag;
»·······uint8_t port_id;

»·······signal(SIGINT, signal_handler);
»·······signal(SIGTERM, signal_handler);

»·······diag = rte_eal_init(argc, argv);
»·······if (diag < 0)
»·······»·······rte_panic("Cannot init EAL\n");
#ifdef RTE_LIBRTE_PDUMP
»·······/* initialize packet capture framework */
»·······rte_pdump_init(NULL);

static void
signal_handler(int signum)
{
»·······if (signum == SIGINT || signum == SIGTERM) {
»·······»·······printf("\nSignal %d received, preparing to exit...\n",
»·······»·······»·······»·······signum);
#ifdef RTE_LIBRTE_PDUMP
»·······»·······/* uninitialize packet capture framework */
»·······»·······rte_pdump_uninit();
#endif

3)As of now initializing and un initializing of the packet capture framework is done only in testpmd, for other applications this is not done. We left users to do this for now.


Please feel free to ping me if you see some more issues.

Thanks,
Reshma
From: Nishant Verma [mailto:vnish11 at gmail.com<mailto:vnish11 at gmail.com>]
Sent: Friday, July 22, 2016 10:42 PM

To: Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>>
Cc: users at dpdk.org<mailto:users at dpdk.org>
Subject: Re: pdump help

Hi,
Something is still creating the problem. just can't figure out.
Here is the scenario....
Just inserted IGB UIO module, attached one 10G NIC to DPDK &
4096 hugepage.

---
trying to run l3fwd as primary and pducp as secondary  to capture packet dump.
Earlier dpdk_pdump is crashing on either create_mp_ring_vdev:649 or create_mp_ring_vdev:669.
Now i changed some parameters of pdump and l3fwd, change hugepages to 8192 and tried to run like this..
./l3fwd -c 606 -n 4 --proc-type=primary  --socket-mem=1024 --file-prefix l3f -- -p 0x1 --config="(0,0,1),(0,1,2)"
Here is the log of app
"
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: VFIO support initialized
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL: PCI device 0000:05:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
L3FWD: LPM or EM none selected, default LPM on
Initializing port 0 ... Creating queues: nb_rxq=2 nb_txq=4...  Address:0C:C4:7A:73:EF:14, Destination:02:00:00:00:00:00, Allocated mbuf pool on socket 0
LPM: Adding route 0x01010100 / 24 (0)
LPM: Adding route IPV6 / 48 (0)
txq=1,0,0 txq=2,1,0 txq=9,2,0 txq=10,3,0

Skipping disabled port 1

Initializing rx queues on lcore 1 ... rxq=0,0,0
Initializing rx queues on lcore 2 ... rxq=0,1,0
Initializing rx queues on lcore 9 ...
Initializing rx queues on lcore 10 ...

"
It seems everything is good in this. So I tried to run pdump like

 ./dpdk_pdump -- --pdump 'port=0,queue=*,rx-dev=/tmp/rx-file.pcap,tx-dev=/tmp/tx-file.pcap,ring-size=1024,mbuf-size=20,total-num-mbufs=1025'
Here is log for pdump

"
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL:   cannot connect to primary process!
EAL: VFIO support could not be initialized
EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the kernel.
EAL:    This may cause issues with mapping memory into secondary processes
EAL: Could not open /dev/hugepages/rtemap_510
PANIC in rte_eal_init():
Cannot init memory
6: [./dpdk_pdump(_start+0x29) [0x4289b9]]
5: [/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0) [0x7fa22219aa40]]
4: [./dpdk_pdump(main+0x155) [0x42c417]]
3: [./dpdk_pdump(rte_eal_init+0xf14) [0x4603b4]]
2: [./dpdk_pdump(__rte_panic+0xc9) [0x424b59]]
1: [./dpdk_pdump(rte_dump_stack+0x2b) [0x4677fb]]
Aborted (core dumped)

"

It seems from log that their is some issue in memory between primary and secondary, but when i tried to run one l2fwd as primary and one l3fwd as secondary it works fine.

On Fri, Jul 22, 2016 at 7:21 AM, Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>> wrote:
Hi,

I see some cleanup related bug in the dpdk_pdump tool, so until I publish fix for that,
You need to restart  both primary and secondary process( work around) after you see any RING/VDEV failure issues.

Thanks,
Reshma


From: Pattan, Reshma
Sent: Friday, July 22, 2016 10:31 AM
To: 'Nishant Verma' <vnish11 at gmail.com<mailto:vnish11 at gmail.com>>
Cc: users at dpdk.org<mailto:users at dpdk.org>
Subject: RE: pdump help

Hi,

As I see “RING: Cannot reserve memory”. Please allocate enough huge page memory and see what happens.

Also once you encounter such error please restart  both primary and secondary process.

Please paste me the full  error log, if you still face the issue.

Thanks,
Reshma


From: Nishant Verma [mailto:vnish11 at gmail.com]
Sent: Thursday, July 21, 2016 9:10 PM
To: Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>>
Cc: users at dpdk.org<mailto:users at dpdk.org>
Subject: Re: pdump help

Hi Reshma,

Tried things pointed by you(libpcap Installed & config parameter is set).
But still things are not working.

"
root at ubuntu:/home/oneweb/dpdk/app/pdump# ./dpdk_pdump --proc-type=secondary -- --pdump 'device_id=03:00.0,queue=*,rx-dev=/tmp/rx-file.pcap,tx-dev=/tmp/tx-file.pcap'
EAL: Detected 16 lcore(s)
EAL: Probing VFIO support...
EAL: WARNING: Address Space Layout Randomization (ASLR) is enabled in the kernel.
EAL:    This may cause issues with mapping memory into secondary processes
PMD: bnxt_rte_pmd_init() called for (null)
EAL: PCI device 0000:03:00.0 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
EAL: PCI device 0000:03:00.1 on NUMA socket 0
EAL:   probe driver: 8086:15ad rte_ixgbe_pmd
PMD: eth_ixgbe_dev_init(): No TX queues configured yet. Using default TX function.
EAL: PCI device 0000:05:00.0 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
EAL: PCI device 0000:05:00.1 on NUMA socket 0
EAL:   probe driver: 8086:1521 rte_igb_pmd
RING: Cannot reserve memory
EAL: Error - exiting with code: 1
  Cause: File exists:create_mp_ring_vdev:649
"


Rgds,
Nishant


On Thu, Jul 21, 2016 at 5:16 AM, Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>> wrote:
Hi,

Pdump tool is designed based on PCAP PMD. So you need to install libpcap package and set CONFIG_RTE_LIBRTE_PMD_PCAP flag to “y”
in config/common_base file and recompile the DPDK and run pdump tool. Please let me know if that fix the issue.

This dependency is mentioned in the  document “doc/guides/sample_app_ug/pdump.rst.”

Thanks,
Reshma

From: Nishant Verma [mailto:vnish11 at gmail.com<mailto:vnish11 at gmail.com>]
Sent: Wednesday, July 20, 2016 8:08 PM
To: Pattan, Reshma <reshma.pattan at intel.com<mailto:reshma.pattan at intel.com>>
Subject: pdump help

Hi Reshma,

i am facing some issue in executing pdump. I have seen in logs that you recently updated the code. Please let me know if i am doing anything wrong.


app/pdump# ./dpdk_pdump -- --pdump 'device_id=03:00.0,queue=*,rx-dev=/tmp/rx-1-file.pcap,tx-dev=tx-1-file.pcap'

This is the error i am getting.
Cause: File exists:create_mp_ring_vdev:649

Thanks a lot.

--
Rgds,
Nishant






--
Rgds,
Nishant






--
Rgds,
Nishant






--
Rgds,
Nishant






--
Rgds,
Nishant
Previous message: [dpdk-users] pdump help
Next message: [dpdk-users] Possible rte_ring issues
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
More information about the users mailing list