[v4] eal: add generic thread-local-storage functions

Message ID 20201222073044.9084-1-talshn@nvidia.com (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers
Series [v4] eal: add generic thread-local-storage functions |

Checks

Context Check Description
ci/checkpatch success coding style OK
ci/Intel-compilation success Compilation OK
ci/iol-broadcom-Functional success Functional Testing PASS
ci/iol-broadcom-Performance success Performance Testing PASS
ci/iol-intel-Performance success Performance Testing PASS
ci/iol-intel-Functional success Functional Testing PASS
ci/iol-abi-testing success Testing PASS
ci/iol-testing success Testing PASS

Commit Message

Tal Shnaiderman Dec. 22, 2020, 7:30 a.m. UTC
  Add support for tls functionality in EAL.

The following functions are added:
rte_thread_tls_create_key - function to create a tls data key.
rte_thread_tls_delete_key - function to delete a tls data key.
rte_thread_tls_set_value - function to set value bound to the tls key
rte_thread_tls_get_value - function to get value bound to the tls key

tls key will be defined by the new type rte_tls_key

Windows implementation is under librte_eal/windows and
implemented using WIN32 API for Windows only.

common implementation is under librte_eal/common and
implemented using pthread for UNIX and Windows compilation
using extenral pthread libraries, when supported.

Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
---
v3: switch from pthread shim to generic eal implementation [DmitryK]
v4: modify file names, function names, move unix code to common
for future external pthreads support [DmitryK]
---
 lib/librte_eal/common/meson.build   |  1 +
 lib/librte_eal/common/rte_thread.c  | 87 ++++++++++++++++++++++++++++++++++++
 lib/librte_eal/include/meson.build  |  1 +
 lib/librte_eal/include/rte_thread.h | 88 +++++++++++++++++++++++++++++++++++++
 lib/librte_eal/rte_eal_exports.def  |  5 +++
 lib/librte_eal/version.map          |  6 +++
 lib/librte_eal/windows/meson.build  |  6 +++
 lib/librte_eal/windows/rte_thread.c | 82 ++++++++++++++++++++++++++++++++++
 8 files changed, 276 insertions(+)
 create mode 100644 lib/librte_eal/common/rte_thread.c
 create mode 100644 lib/librte_eal/include/rte_thread.h
 create mode 100644 lib/librte_eal/windows/rte_thread.c
  

Comments

Dmitry Kozlyuk Dec. 23, 2020, 1:18 a.m. UTC | #1
> [...]
> +int
> +rte_thread_tls_set_value(rte_tls_key key, const void *value)
> +{
> +	int err;
> +
> +	if (!key) {
> +		RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
> +		return -1;
> +	}
> +	err = pthread_setspecific(key->thread_index, value);
> +	if (err) {
> +		RTE_LOG(DEBUG, EAL, "pthread_setspecific failed: %s\n",
> +			strerror(err));
> +		free(key);

Why free(key) here? Probably a typo.

> [...]
> +__rte_experimental
> +int
> +rte_thread_tls_create_key(rte_tls_key *key, void (*destructor)(void *));
> +
> +/**
> + * Function to delete a TLS data key visible to all threads in the process
> + * rte_tls_key is the opaque pointer allocated by rte_thread_tls_create_key.
> + *
> + * @param key
> + *   The rte_tls_key will cantain the allocated key

cantain -> contain


> diff --git a/lib/librte_eal/windows/meson.build b/lib/librte_eal/windows/meson.build
> index 3b2faf29eb..1f1398dfe9 100644
> --- a/lib/librte_eal/windows/meson.build
> +++ b/lib/librte_eal/windows/meson.build
> @@ -21,4 +21,10 @@ sources += files(
>  	'getopt.c',
>  )
>  
> +if (dpdk_conf.has('use_external_pthread'))

Please describe the new option in meson_options.txt.
Maybe drop "external" from the name, what do you think?
  
Tal Shnaiderman Dec. 23, 2020, 11:44 a.m. UTC | #2
> Subject: Re: [PATCH v4] eal: add generic thread-local-storage functions
> 
> External email: Use caution opening links or attachments
> 
> 
> > [...]
> > +int
> > +rte_thread_tls_set_value(rte_tls_key key, const void *value) {
> > +     int err;
> > +
> > +     if (!key) {
> > +             RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
> > +             return -1;
> > +     }
> > +     err = pthread_setspecific(key->thread_index, value);
> > +     if (err) {
> > +             RTE_LOG(DEBUG, EAL, "pthread_setspecific failed: %s\n",
> > +                     strerror(err));
> > +             free(key);
> 
> Why free(key) here? Probably a typo.
> 
> > [...]
> > +__rte_experimental
> > +int
> > +rte_thread_tls_create_key(rte_tls_key *key, void (*destructor)(void
> > +*));
> > +
> > +/**
> > + * Function to delete a TLS data key visible to all threads in the
> > +process
> > + * rte_tls_key is the opaque pointer allocated by
> rte_thread_tls_create_key.
> > + *
> > + * @param key
> > + *   The rte_tls_key will cantain the allocated key
> 
> cantain -> contain
> 
> 
> > diff --git a/lib/librte_eal/windows/meson.build
> > b/lib/librte_eal/windows/meson.build
> > index 3b2faf29eb..1f1398dfe9 100644
> > --- a/lib/librte_eal/windows/meson.build
> > +++ b/lib/librte_eal/windows/meson.build
> > @@ -21,4 +21,10 @@ sources += files(
> >       'getopt.c',
> >  )
> >
> > +if (dpdk_conf.has('use_external_pthread'))
> 
> Please describe the new option in meson_options.txt.
> Maybe drop "external" from the name, what do you think?

How about use_windows_pthread to be clear that it's unrelated to UNIX pthreads?
  
Dmitry Kozlyuk Dec. 23, 2020, 11:58 a.m. UTC | #3
On Wed, 23 Dec 2020 11:44:31 +0000, Tal Shnaiderman wrote:
> > > diff --git a/lib/librte_eal/windows/meson.build
> > > b/lib/librte_eal/windows/meson.build
> > > index 3b2faf29eb..1f1398dfe9 100644
> > > --- a/lib/librte_eal/windows/meson.build
> > > +++ b/lib/librte_eal/windows/meson.build
> > > @@ -21,4 +21,10 @@ sources += files(
> > >       'getopt.c',
> > >  )
> > >
> > > +if (dpdk_conf.has('use_external_pthread'))  
> > 
> > Please describe the new option in meson_options.txt.
> > Maybe drop "external" from the name, what do you think?  
> 
> How about use_windows_pthread to be clear that it's unrelated to UNIX pthreads?

Sounds good.
  
Dmitry Malloy Dec. 23, 2020, 6:16 p.m. UTC | #4
I wonder if this should be consistent with Nick's meson (to use external pthread library) change?

-----Original Message-----
From: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com> 
Sent: Wednesday, December 23, 2020 3:59 AM
To: Tal Shnaiderman <talshn@nvidia.com>
Cc: dev@dpdk.org; thomas <thomas@monjalon.net>; Kadam, Pallavi <pallavi.kadam@intel.com>; navasile@linux.microsoft.com; Dmitry Malloy (MESHCHANINOV) <dmitrym@microsoft.com>; david.marchand@redhat.com
Subject: [EXTERNAL] Re: [PATCH v4] eal: add generic thread-local-storage functions

On Wed, 23 Dec 2020 11:44:31 +0000, Tal Shnaiderman wrote:
> > > diff --git a/lib/librte_eal/windows/meson.build
> > > b/lib/librte_eal/windows/meson.build
> > > index 3b2faf29eb..1f1398dfe9 100644
> > > --- a/lib/librte_eal/windows/meson.build
> > > +++ b/lib/librte_eal/windows/meson.build
> > > @@ -21,4 +21,10 @@ sources += files(
> > >       'getopt.c',
> > >  )
> > >
> > > +if (dpdk_conf.has('use_external_pthread'))  
> > 
> > Please describe the new option in meson_options.txt.
> > Maybe drop "external" from the name, what do you think?  
> 
> How about use_windows_pthread to be clear that it's unrelated to UNIX pthreads?

Sounds good.
  

Patch

diff --git a/lib/librte_eal/common/meson.build b/lib/librte_eal/common/meson.build
index 39abf7a0a4..69c88b3349 100644
--- a/lib/librte_eal/common/meson.build
+++ b/lib/librte_eal/common/meson.build
@@ -77,6 +77,7 @@  sources += files(
 	'rte_random.c',
 	'rte_reciprocal.c',
 	'rte_service.c',
+	'rte_thread.c',
 )
 
 if is_linux
diff --git a/lib/librte_eal/common/rte_thread.c b/lib/librte_eal/common/rte_thread.c
new file mode 100644
index 0000000000..a2923a4ad1
--- /dev/null
+++ b/lib/librte_eal/common/rte_thread.c
@@ -0,0 +1,87 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+
+#include <errno.h>
+#include <pthread.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include <rte_common.h>
+#include <rte_errno.h>
+#include <rte_log.h>
+#include <rte_thread.h>
+
+struct eal_tls_key {
+	pthread_key_t thread_index;
+};
+
+int
+rte_thread_tls_create_key(rte_tls_key *key, void (*destructor)(void *))
+{
+	int err;
+
+	*key = malloc(sizeof(struct eal_tls_key));
+	if ((*key) == NULL) {
+		RTE_LOG(DEBUG, EAL, "Cannot allocate tls key.");
+		return -1;
+	}
+	err = pthread_key_create(&((*key)->thread_index), destructor);
+	if (err) {
+		RTE_LOG(DEBUG, EAL, "pthread_key_create failed: %s\n",
+			 strerror(err));
+		free(*key);
+		return -1;
+	}
+	return 0;
+}
+
+int
+rte_thread_tls_delete_key(rte_tls_key key)
+{
+	int err;
+
+	if (!key) {
+		RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+		return -1;
+	}
+	err = pthread_key_delete(key->thread_index);
+	if (err) {
+		RTE_LOG(DEBUG, EAL, "pthread_key_delete failed: %s\n",
+			 strerror(err));
+		free(key);
+		return -1;
+	}
+	free(key);
+	return 0;
+}
+
+int
+rte_thread_tls_set_value(rte_tls_key key, const void *value)
+{
+	int err;
+
+	if (!key) {
+		RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+		return -1;
+	}
+	err = pthread_setspecific(key->thread_index, value);
+	if (err) {
+		RTE_LOG(DEBUG, EAL, "pthread_setspecific failed: %s\n",
+			strerror(err));
+		free(key);
+		return -1;
+	}
+	return 0;
+}
+
+void *
+rte_thread_tls_get_value(rte_tls_key key)
+{
+	if (!key) {
+		RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	return pthread_getspecific(key->thread_index);
+}
diff --git a/lib/librte_eal/include/meson.build b/lib/librte_eal/include/meson.build
index dc007084ff..0dea342e1d 100644
--- a/lib/librte_eal/include/meson.build
+++ b/lib/librte_eal/include/meson.build
@@ -40,6 +40,7 @@  headers += files(
 	'rte_service_component.h',
 	'rte_string_fns.h',
 	'rte_tailq.h',
+	'rte_thread.h',
 	'rte_time.h',
 	'rte_trace.h',
 	'rte_trace_point.h',
diff --git a/lib/librte_eal/include/rte_thread.h b/lib/librte_eal/include/rte_thread.h
new file mode 100644
index 0000000000..6686f6183c
--- /dev/null
+++ b/lib/librte_eal/include/rte_thread.h
@@ -0,0 +1,88 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2020 Mellanox Technologies, Ltd
+ */
+
+#include <rte_compat.h>
+
+#ifndef _RTE_THREAD_H_
+#define _RTE_THREAD_H_
+
+/**
+ * @file
+ *
+ * Threading functions
+ *
+ * Simple threads functionality supplied by EAL.
+ */
+
+/**
+ * Opaque pointer for TLS key.
+ */
+typedef struct eal_tls_key *rte_tls_key;
+
+/**
+ * Function to create a TLS data key visible to all threads in the process
+ * function need to be called once to create a key usable by all threads.
+ * rte_tls_key is an opaque pointer used to store the allocated key.
+ * and optional destructor can be set to be called when a thread expires.
+ *
+ * @param key
+ *   Pointer to store the allocated rte_tls_key
+ * @param destructor
+ *   The function to be called when the thread expires.
+ *   Not supported on Windows OS.
+ *
+ * @return
+ *   On success, zero.
+ *   On failure, a negative number.
+ */
+__rte_experimental
+int
+rte_thread_tls_create_key(rte_tls_key *key, void (*destructor)(void *));
+
+/**
+ * Function to delete a TLS data key visible to all threads in the process
+ * rte_tls_key is the opaque pointer allocated by rte_thread_tls_create_key.
+ *
+ * @param key
+ *   The rte_tls_key will cantain the allocated key
+ *
+ * @return
+ *   On success, zero.
+ *   On failure, a negative number.
+ */
+__rte_experimental
+int
+rte_thread_tls_delete_key(rte_tls_key key);
+
+/**
+ * Function to set value bound to the tls key on behalf of the calling thread
+ *
+ * @param key
+ *   The rte_tls_key key allocated by rte_thread_tls_create_key.
+ * @param value
+ *   The value bound to the rte_tls_key key for the calling thread.
+ *
+ * @return
+ *   On success, zero.
+ *   On failure, a negative number.
+ */
+__rte_experimental
+int
+rte_thread_tls_set_value(rte_tls_key key, const void *value);
+
+/**
+ * Function to get value bound to the tls key on behalf of the calling thread
+ *
+ * @param key
+ *   The rte_tls_key key allocated by rte_thread_tls_create_key.
+ *
+ * @return
+ *   On success, value data pointer (can also be NULL).
+ *   On failure, NULL and an error number is set in rte_errno.
+ */
+__rte_experimental
+void *
+rte_thread_tls_get_value(rte_tls_key key);
+
+#endif /* _RTE_THREAD_H_ */
diff --git a/lib/librte_eal/rte_eal_exports.def b/lib/librte_eal/rte_eal_exports.def
index 6a6be1cfa6..a071fe9c2f 100644
--- a/lib/librte_eal/rte_eal_exports.def
+++ b/lib/librte_eal/rte_eal_exports.def
@@ -306,6 +306,11 @@  EXPORTS
 	rte_vect_get_max_simd_bitwidth
 	rte_vect_set_max_simd_bitwidth
 
+	rte_thread_tls_create_key
+	rte_thread_tls_delete_key
+	rte_thread_tls_set_value
+	rte_thread_tls_get_value
+
 	rte_mem_lock
 	rte_mem_map
 	rte_mem_page_size
diff --git a/lib/librte_eal/version.map b/lib/librte_eal/version.map
index 354c068f31..8559694bd5 100644
--- a/lib/librte_eal/version.map
+++ b/lib/librte_eal/version.map
@@ -403,6 +403,12 @@  EXPERIMENTAL {
 	rte_service_lcore_may_be_active;
 	rte_vect_get_max_simd_bitwidth;
 	rte_vect_set_max_simd_bitwidth;
+
+	# added in 21.02
+	rte_thread_tls_create_key;
+	rte_thread_tls_delete_key;
+	rte_thread_tls_set_value;
+	rte_thread_tls_get_value;
 };
 
 INTERNAL {
diff --git a/lib/librte_eal/windows/meson.build b/lib/librte_eal/windows/meson.build
index 3b2faf29eb..1f1398dfe9 100644
--- a/lib/librte_eal/windows/meson.build
+++ b/lib/librte_eal/windows/meson.build
@@ -21,4 +21,10 @@  sources += files(
 	'getopt.c',
 )
 
+if (dpdk_conf.has('use_external_pthread'))
+	sources += 'librte_eal/common/rte_thread.c'
+else
+	sources += 'librte_eal/windows/rte_thread.c'
+endif
+
 dpdk_conf.set10('RTE_EAL_NUMA_AWARE_HUGEPAGES', true)
diff --git a/lib/librte_eal/windows/rte_thread.c b/lib/librte_eal/windows/rte_thread.c
new file mode 100644
index 0000000000..645eca7478
--- /dev/null
+++ b/lib/librte_eal/windows/rte_thread.c
@@ -0,0 +1,82 @@ 
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Mellanox Technologies, Ltd
+ */
+#include <rte_common.h>
+#include <rte_errno.h>
+#include <rte_thread.h>
+#include <rte_windows.h>
+
+struct eal_tls_key {
+	DWORD thread_index;
+};
+
+int
+rte_thread_tls_create_key(rte_tls_key *key,
+		__rte_unused void (*destructor)(void *))
+{
+	*key = malloc(sizeof(struct eal_tls_key));
+	if ((*key) == NULL) {
+		RTE_LOG(DEBUG, EAL, "Cannot allocate tls key.");
+		return -1;
+	}
+	(*key)->thread_index = TlsAlloc();
+	if ((*key)->thread_index == TLS_OUT_OF_INDEXES) {
+		RTE_LOG_WIN32_ERR("TlsAlloc()");
+		free(*key);
+		return -1;
+	}
+	return 0;
+}
+
+int
+rte_thread_tls_delete_key(rte_tls_key key)
+{
+	if (!key) {
+		RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+		return -1;
+	}
+	if (!TlsFree(key->thread_index)) {
+		RTE_LOG_WIN32_ERR("TlsFree()");
+		free(key);
+		return -1;
+	}
+	free(key);
+	return 0;
+}
+
+int
+rte_thread_tls_set_value(rte_tls_key key, const void *value)
+{
+	char *p;
+
+	if (!key) {
+		RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+		return -1;
+	}
+	/* discard const qualifier */
+	p = (char *) (uintptr_t) value;
+	if (!TlsSetValue(key->thread_index, p)) {
+		RTE_LOG_WIN32_ERR("TlsSetValue()");
+		return -1;
+	}
+	return 0;
+}
+
+void *
+rte_thread_tls_get_value(rte_tls_key key)
+{
+	void *output;
+
+	if (!key) {
+		RTE_LOG(DEBUG, EAL, "invalid tls key passed to function.\n");
+		rte_errno = EINVAL;
+		return NULL;
+	}
+	output = TlsGetValue(key->thread_index);
+	if (GetLastError() != ERROR_SUCCESS) {
+		RTE_LOG_WIN32_ERR("TlsGetValue()");
+		rte_errno = ENOEXEC;
+		return NULL;
+	}
+	return output;
+}