diff mbox

[dpdk-dev,4/4] eal: set affinity for control threads

Message ID 20180227144630.29613-5-olivier.matz@6wind.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK

Commit Message

Olivier Matz Feb. 27, 2018, 2:46 p.m. UTC 
  The management threads must not bother the dataplane or service cores.
Set the affinity of these threads accordingly.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---
 lib/librte_eal/common/eal_common_thread.c | 20 +++++++++++++++++++-
 lib/librte_eal/common/include/rte_lcore.h |  4 +++-
 2 files changed, 22 insertions(+), 2 deletions(-)

Comments

Anatoly Burakov March 29, 2018, 8:04 a.m. UTC | #1 
On 27-Feb-18 2:46 PM, Olivier Matz wrote:
> The management threads must not bother the dataplane or service cores.
> Set the affinity of these threads accordingly.
> 
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---

<...>

>   
> +	set_affinity = 0;
> +	CPU_ZERO(&cpuset);
> +	for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> +		if (eal_cpu_detected(lcore_id) &&
> +				rte_lcore_has_role(lcore_id, ROLE_OFF)) {
> +			CPU_SET(lcore_id, &cpuset);
> +			set_affinity = 1;
> +		}
> +	}
> +	if (set_affinity) {
> +		ret = pthread_setaffinity_np(*thread, sizeof(cpuset), &cpuset);
> +		if (ret < 0)
> +			goto fail;
> +	}

Hi Olivier,

Please correct me if i'm wrong here, but if all detected cores are busy 
doing something (such as would be the case if DPDK is run without a 
coremask specified), affinity is not set? Maybe set it to master lcore 
instead (perhaps unconditionally - we usually recommend not using master 
lcore for anything important - perhaps setting it to master core always 
would be better, instead of trying to find unused lcores)?
Olivier Matz March 30, 2018, 7:30 a.m. UTC | #2 
On Thu, Mar 29, 2018 at 09:04:52AM +0100, Burakov, Anatoly wrote:
> On 27-Feb-18 2:46 PM, Olivier Matz wrote:
> > The management threads must not bother the dataplane or service cores.
> > Set the affinity of these threads accordingly.
> > 
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > ---
> 
> <...>
> 
> > +	set_affinity = 0;
> > +	CPU_ZERO(&cpuset);
> > +	for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
> > +		if (eal_cpu_detected(lcore_id) &&
> > +				rte_lcore_has_role(lcore_id, ROLE_OFF)) {
> > +			CPU_SET(lcore_id, &cpuset);
> > +			set_affinity = 1;
> > +		}
> > +	}
> > +	if (set_affinity) {
> > +		ret = pthread_setaffinity_np(*thread, sizeof(cpuset), &cpuset);
> > +		if (ret < 0)
> > +			goto fail;
> > +	}
> 
> Hi Olivier,
> 
> Please correct me if i'm wrong here, but if all detected cores are busy
> doing something (such as would be the case if DPDK is run without a coremask
> specified), affinity is not set? Maybe set it to master lcore instead
> (perhaps unconditionally - we usually recommend not using master lcore for
> anything important - perhaps setting it to master core always would be
> better, instead of trying to find unused lcores)?

Good point and good idea, I will update the patchset.

Thanks,
Olivier
diff mbox

Patch

diff --git a/lib/librte_eal/common/eal_common_thread.c b/lib/librte_eal/common/eal_common_thread.c
index 575b03e9d..f2e588c97 100644
--- a/lib/librte_eal/common/eal_common_thread.c
+++ b/lib/librte_eal/common/eal_common_thread.c
@@ -16,6 +16,7 @@ 
 #include <rte_memory.h>
 #include <rte_log.h>
 
+#include "eal_private.h"
 #include "eal_thread.h"
 
 RTE_DECLARE_PER_LCORE(unsigned , _socket_id);
@@ -169,7 +170,9 @@  rte_ctrl_thread_create(pthread_t *thread, const char *name,
 		.start_routine = start_routine,
 		.arg = arg,
 	};
-	int ret;
+	unsigned int lcore_id;
+	rte_cpuset_t cpuset;
+	int set_affinity, ret;
 
 	pthread_barrier_init(&params.configured, NULL, 2);
 
@@ -183,6 +186,21 @@  rte_ctrl_thread_create(pthread_t *thread, const char *name,
 			goto fail;
 	}
 
+	set_affinity = 0;
+	CPU_ZERO(&cpuset);
+	for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+		if (eal_cpu_detected(lcore_id) &&
+				rte_lcore_has_role(lcore_id, ROLE_OFF)) {
+			CPU_SET(lcore_id, &cpuset);
+			set_affinity = 1;
+		}
+	}
+	if (set_affinity) {
+		ret = pthread_setaffinity_np(*thread, sizeof(cpuset), &cpuset);
+		if (ret < 0)
+			goto fail;
+	}
+
 	pthread_barrier_wait(&params.configured);
 
 	return 0;
diff --git a/lib/librte_eal/common/include/rte_lcore.h b/lib/librte_eal/common/include/rte_lcore.h
index f3d9bbb91..354717c5d 100644
--- a/lib/librte_eal/common/include/rte_lcore.h
+++ b/lib/librte_eal/common/include/rte_lcore.h
@@ -249,7 +249,9 @@  int rte_thread_setname(pthread_t id, const char *name);
 /**
  * Create a control thread.
  *
- * Wrapper to pthread_create() and pthread_setname_np().
+ * Wrapper to pthread_create(), pthread_setname_np() and
+ * pthread_setaffinity_np(). The dataplane and service lcores are
+ * excluded from the affinity of the new thread.
  *
  * @param thread
  *   Filled with the thread id of the new created thread.