[dpdk-dev] Order of system brought up affects throughput with qos_sched app

[Wei Shen]

Hi all,
I ran into problems with qos_sched with different order of system brought up. I can bring up the system in two ways:
1. Start traffic gen first. Then start qos_sched.2. Start qos_sched first. Then start traffic gen.
With 256K pipes and 64 queue size, 128B packet size, I got ~4Gbps with order #1. While I got 10G with order #2.
qos_sched command stats showed that ~59% packets got dropped in RX (rte_ring_enqueue).
Plus, with #1, if I restart the traffic gen later, I would regain 10Gbps throughput, which suggests that this is not an initialization issue but runtime behavior.
I also tried to assign qos_sched on different cores and got the same result.
I suspect that there is some rte_ring bugs when connecting two cores and one core started enqueuing before another core is ready for dequeue.
Have you experienced the same issue? Appreciate your help.

Wei Shen.------------------------------------------------------------------------------------------------------------------------------------My system spec is:Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz15 * 1G hugepages
qos_sched argument: ./build/app/qos_sched -c 1c0002 -n 4 -- --pfc "0,1,20,18,19" --cfg profile.cfg
profile.cfg:[port]frame overhead = 20number of subports per port = 1number of pipes per subport = 262144queue sizes = 64 64 64 64

; Subport configuration[subport 0]tb rate = 1250000000           ; Bytes per secondtb size = 1000000                 ; Bytes
tc 0 rate = 1250000000         ; Bytes per secondtc 1 rate = 1250000000         ; Bytes per secondtc 2 rate = 1250000000         ; Bytes per secondtc 3 rate = 1250000000         ; Bytes per secondtc period = 10                        ; Milliseconds
pipe 0-262143 = 0                ; These pipes are configured with pipe profile 0
; Pipe configuration[pipe profile 0]tb rate = 1250000000           ; Bytes per secondtb size = 1000000                 ; Bytes
tc 0 rate = 1250000000         ; Bytes per secondtc 1 rate = 1250000000         ; Bytes per secondtc 2 rate = 1250000000         ; Bytes per secondtc 3 rate = 1250000000         ; Bytes per secondtc period = 10                        ; Milliseconds
tc 3 oversubscription weight = 1
tc 0 wrr weights = 1 1 1 1tc 1 wrr weights = 1 1 1 1tc 2 wrr weights = 1 1 1 1tc 3 wrr weights = 1 1 1 1