[dpdk-users] Using DPDK EAL with existing multi-threaded application

[Jason Lefley]

Hi Daniel,

Thanks for the information. I reverted my changes to rte_eal_init() and added the following code

rte_cpuset_t cpuSet;
CPU_ZERO(&cpuSet);
unsigned coreCount = sysconf(_SC_NPROCESSORS_CONF);
for (unsigned i = 0; i < coreCount; i++)
{
    CPU_SET(i, &cpuSet);
}
rte_thread_set_affinity(&cpuSet);

after calling rte_eal_init(). My TBB code now works as expected.

> On Nov 11, 2016, at 10:19 AM, Daniel Verkamp <daniel.verkamp at intel.com> wrote:
> 
> On 11/07/2016 11:39 PM, Jason Lefley wrote:
>> I am working on leveraging SPDK (which internally uses EAL from DPDK)
>> from an existing application that uses Intel’s TBB. I do not need to
>> use EAL for multithreading but SPDK does require at least huge page
>> support and PCI access from EAL.
>> 
>> I noticed that if I try to use TBB’s parallel_for() after I have
>> called rte_eal_init(), then the TBB operation doesn’t actually run in
>> parallel. If I remove the call to rte_eal_init(), the TBB operations
>> work as expected however I then cannot use SPDK. Does anyone have any
>> input regarding how I can keep my existing multi-threaded application
>> working as intended as I integrate SPDK/DPDK?
>> 
>> Thanks
>> 
> 
> Hi Jason,
> 
> If you are not calling any functions that rely on DPDK's lcore pinning
> for correctness, you should be able to change the calling thread's CPU
> affinity with rte_thread_set_affinity() so that newly-created threads
> can run on more than one CPU core.  I am not sure how TBB creates its
> threads, but as long as you set the affinity before calling any TBB
> functions, the updated CPU affinity should be inherited by any
> newly-created threads.
> 
> There is a recently-implemented helper function in SPDK to temporarily
> change the thread affinity in this way:
> 
> https://github.com/spdk/spdk/commit/fa5206c416144b41c384d678cf33f98f1fe1042c
> 
> Thanks,
> -- Daniel