From patchwork Wed Sep 13 07:22:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "De Lara Guarch, Pablo" X-Patchwork-Id: 28692 X-Patchwork-Delegate: pablo.de.lara.guarch@intel.com Return-Path: X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id 3A7861B1A7; Wed, 13 Sep 2017 17:22:41 +0200 (CEST) Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by dpdk.org (Postfix) with ESMTP id 147F41B1C0 for ; Wed, 13 Sep 2017 17:22:38 +0200 (CEST) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Sep 2017 08:22:38 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.42,388,1500966000"; d="scan'208";a="151419346" Received: from silpixa00399464.ir.intel.com (HELO silpixa00399464.ger.corp.intel.com) ([10.237.222.157]) by fmsmga006.fm.intel.com with ESMTP; 13 Sep 2017 08:22:36 -0700 From: Pablo de Lara To: declan.doherty@intel.com, akhil.goyal@nxp.com, hemant.agrawal@nxp.com, jerin.jacob@caviumnetworks.com, fiona.trahe@intel.com, deepak.k.jain@intel.com, john.griffin@intel.com Cc: dev@dpdk.org, Pablo de Lara Date: Wed, 13 Sep 2017 08:22:48 +0100 Message-Id: <20170913072249.29797-7-pablo.de.lara.guarch@intel.com> X-Mailer: git-send-email 2.9.4 In-Reply-To: <20170913072249.29797-1-pablo.de.lara.guarch@intel.com> References: <20170818080520.43088-1-pablo.de.lara.guarch@intel.com> <20170913072249.29797-1-pablo.de.lara.guarch@intel.com> Subject: [dpdk-dev] [PATCH v2 6/7] app/crypto-perf: support multiple queue pairs X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Add parameter "qps" in crypto performance app, to create multiple queue pairs per device. This new parameter is useful to have multiple logical cores using a single crypto device, without needing to initialize a crypto device per core. Signed-off-by: Pablo de Lara --- app/test-crypto-perf/cperf_options.h | 2 + app/test-crypto-perf/cperf_options_parsing.c | 22 +++++++++++ app/test-crypto-perf/cperf_test_latency.c | 14 +++---- app/test-crypto-perf/cperf_test_throughput.c | 14 +++---- app/test-crypto-perf/cperf_test_verify.c | 14 +++---- app/test-crypto-perf/main.c | 56 +++++++++++++++++----------- doc/guides/tools/cryptoperf.rst | 5 +++ 7 files changed, 81 insertions(+), 46 deletions(-) diff --git a/app/test-crypto-perf/cperf_options.h b/app/test-crypto-perf/cperf_options.h index 5f2b28b..362f762 100644 --- a/app/test-crypto-perf/cperf_options.h +++ b/app/test-crypto-perf/cperf_options.h @@ -14,6 +14,7 @@ #define CPERF_SEGMENT_SIZE ("segment-sz") #define CPERF_DEVTYPE ("devtype") +#define CPERF_NUM_QPS ("qps") #define CPERF_OPTYPE ("optype") #define CPERF_SESSIONLESS ("sessionless") #define CPERF_OUT_OF_PLACE ("out-of-place") @@ -67,6 +68,7 @@ struct cperf_options { uint32_t pool_sz; uint32_t total_ops; uint32_t segment_sz; + uint32_t num_qps; uint32_t test_buffer_size; uint32_t sessionless:1; diff --git a/app/test-crypto-perf/cperf_options_parsing.c b/app/test-crypto-perf/cperf_options_parsing.c index 52b884f..d4f2c31 100644 --- a/app/test-crypto-perf/cperf_options_parsing.c +++ b/app/test-crypto-perf/cperf_options_parsing.c @@ -342,6 +342,24 @@ parse_segment_sz(struct cperf_options *opts, const char *arg) } static int +parse_num_qps(struct cperf_options *opts, const char *arg) +{ + int ret = parse_uint32_t(&opts->num_qps, arg); + + if (ret) { + RTE_LOG(ERR, USER1, "failed to parse number of queue pairs\n"); + return -1; + } + + if ((opts->num_qps == 0) || (opts->num_qps > 256)) { + RTE_LOG(ERR, USER1, "invalid number of queue pairs specified\n"); + return -1; + } + + return 0; +} + +static int parse_device_type(struct cperf_options *opts, const char *arg) { if (strlen(arg) > (sizeof(opts->device_type) - 1)) @@ -645,6 +663,7 @@ static struct option lgopts[] = { { CPERF_SEGMENT_SIZE, required_argument, 0, 0 }, { CPERF_DEVTYPE, required_argument, 0, 0 }, + { CPERF_NUM_QPS, required_argument, 0, 0 }, { CPERF_OPTYPE, required_argument, 0, 0 }, { CPERF_SILENT, no_argument, 0, 0 }, @@ -707,6 +726,7 @@ cperf_options_default(struct cperf_options *opts) strncpy(opts->device_type, "crypto_aesni_mb", sizeof(opts->device_type)); + opts->num_qps = 1; opts->op_type = CPERF_CIPHER_THEN_AUTH; @@ -747,6 +767,7 @@ cperf_opts_parse_long(int opt_idx, struct cperf_options *opts) { CPERF_BUFFER_SIZE, parse_buffer_sz }, { CPERF_SEGMENT_SIZE, parse_segment_sz }, { CPERF_DEVTYPE, parse_device_type }, + { CPERF_NUM_QPS, parse_num_qps }, { CPERF_OPTYPE, parse_op_type }, { CPERF_SESSIONLESS, parse_sessionless }, { CPERF_OUT_OF_PLACE, parse_out_of_place }, @@ -980,6 +1001,7 @@ cperf_options_dump(struct cperf_options *opts) printf("#\n"); printf("# cryptodev type: %s\n", opts->device_type); printf("#\n"); + printf("# number of queue pairs per device: %u\n", opts->num_qps); printf("# crypto operation: %s\n", cperf_op_type_strs[opts->op_type]); printf("# sessionless: %s\n", opts->sessionless ? "yes" : "no"); printf("# out of place: %s\n", opts->out_of_place ? "yes" : "no"); diff --git a/app/test-crypto-perf/cperf_test_latency.c b/app/test-crypto-perf/cperf_test_latency.c index 997844a..ae016a6 100644 --- a/app/test-crypto-perf/cperf_test_latency.c +++ b/app/test-crypto-perf/cperf_test_latency.c @@ -218,8 +218,8 @@ cperf_latency_test_constructor(struct rte_mempool *sess_mp, if (ctx->sess == NULL) goto err; - snprintf(pool_name, sizeof(pool_name), "cperf_pool_in_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_pool_in_cdev_%d_qp_%d", + dev_id, qp_id); uint32_t max_size = options->max_buffer_size + options->digest_sz; uint32_t segment_nb = (max_size % options->segment_sz) ? @@ -252,8 +252,8 @@ cperf_latency_test_constructor(struct rte_mempool *sess_mp, if (options->out_of_place == 1) { snprintf(pool_name, sizeof(pool_name), - "cperf_pool_out_cdev_%d", - dev_id); + "cperf_pool_out_cdev_%d_qp_%d", + dev_id, qp_id); ctx->pkt_mbuf_pool_out = rte_pktmbuf_pool_create( pool_name, options->pool_sz, 0, 0, @@ -281,8 +281,8 @@ cperf_latency_test_constructor(struct rte_mempool *sess_mp, } } - snprintf(pool_name, sizeof(pool_name), "cperf_op_pool_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_op_pool_cdev_%d_qp_%d", + dev_id, qp_id); uint16_t priv_size = RTE_ALIGN_CEIL(sizeof(struct priv_op_data) + test_vector->cipher_iv.length + @@ -583,7 +583,5 @@ cperf_latency_test_destructor(void *arg) if (ctx == NULL) return; - rte_cryptodev_stop(ctx->dev_id); - cperf_latency_test_free(ctx, ctx->options->pool_sz); } diff --git a/app/test-crypto-perf/cperf_test_throughput.c b/app/test-crypto-perf/cperf_test_throughput.c index 121ceb1..57d9245 100644 --- a/app/test-crypto-perf/cperf_test_throughput.c +++ b/app/test-crypto-perf/cperf_test_throughput.c @@ -201,8 +201,8 @@ cperf_throughput_test_constructor(struct rte_mempool *sess_mp, if (ctx->sess == NULL) goto err; - snprintf(pool_name, sizeof(pool_name), "cperf_pool_in_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_pool_in_cdev_%d_qp_%d", + dev_id, qp_id); uint32_t max_size = options->max_buffer_size + options->digest_sz; uint32_t segment_nb = (max_size % options->segment_sz) ? @@ -233,8 +233,8 @@ cperf_throughput_test_constructor(struct rte_mempool *sess_mp, if (options->out_of_place == 1) { - snprintf(pool_name, sizeof(pool_name), "cperf_pool_out_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_pool_out_cdev_%d_qp_%d", + dev_id, qp_id); ctx->pkt_mbuf_pool_out = rte_pktmbuf_pool_create( pool_name, options->pool_sz, 0, 0, @@ -262,8 +262,8 @@ cperf_throughput_test_constructor(struct rte_mempool *sess_mp, } } - snprintf(pool_name, sizeof(pool_name), "cperf_op_pool_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_op_pool_cdev_%d_qp_%d", + dev_id, qp_id); uint16_t priv_size = RTE_ALIGN_CEIL(test_vector->cipher_iv.length + test_vector->auth_iv.length + test_vector->aead_iv.length, 16) + @@ -530,7 +530,5 @@ cperf_throughput_test_destructor(void *arg) if (ctx == NULL) return; - rte_cryptodev_stop(ctx->dev_id); - cperf_throughput_test_free(ctx, ctx->options->pool_sz); } diff --git a/app/test-crypto-perf/cperf_test_verify.c b/app/test-crypto-perf/cperf_test_verify.c index b18426c..c7c59d4 100644 --- a/app/test-crypto-perf/cperf_test_verify.c +++ b/app/test-crypto-perf/cperf_test_verify.c @@ -233,8 +233,8 @@ cperf_verify_test_constructor(struct rte_mempool *sess_mp, if (ctx->sess == NULL) goto err; - snprintf(pool_name, sizeof(pool_name), "cperf_pool_in_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_pool_in_cdev_%d_qp_%d", + dev_id, qp_id); uint32_t max_size = options->max_buffer_size + options->digest_sz; uint32_t segment_nb = (max_size % options->segment_sz) ? @@ -265,8 +265,8 @@ cperf_verify_test_constructor(struct rte_mempool *sess_mp, if (options->out_of_place == 1) { - snprintf(pool_name, sizeof(pool_name), "cperf_pool_out_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_pool_out_cdev_%d_qp_%d", + dev_id, qp_id); ctx->pkt_mbuf_pool_out = rte_pktmbuf_pool_create( pool_name, options->pool_sz, 0, 0, @@ -294,8 +294,8 @@ cperf_verify_test_constructor(struct rte_mempool *sess_mp, } } - snprintf(pool_name, sizeof(pool_name), "cperf_op_pool_cdev_%d", - dev_id); + snprintf(pool_name, sizeof(pool_name), "cperf_op_pool_cdev_%d_qp_%d", + dev_id, qp_id); uint16_t priv_size = RTE_ALIGN_CEIL(test_vector->cipher_iv.length + test_vector->auth_iv.length + test_vector->aead_iv.length, 16) + @@ -626,7 +626,5 @@ cperf_verify_test_destructor(void *arg) if (ctx == NULL) return; - rte_cryptodev_stop(ctx->dev_id); - cperf_verify_test_free(ctx, ctx->options->pool_sz); } diff --git a/app/test-crypto-perf/main.c b/app/test-crypto-perf/main.c index 99f5d3e..4641a22 100644 --- a/app/test-crypto-perf/main.c +++ b/app/test-crypto-perf/main.c @@ -83,7 +83,7 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs, struct rte_mempool *session_pool_socket[]) { uint8_t enabled_cdev_count = 0, nb_lcores, cdev_id; - unsigned int i; + unsigned int i, j; int ret; enabled_cdev_count = rte_cryptodev_devices_get(opts->device_type, @@ -118,8 +118,8 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs, uint8_t socket_id = rte_cryptodev_socket_id(cdev_id); struct rte_cryptodev_config conf = { - .nb_queue_pairs = 1, - .socket_id = socket_id + .nb_queue_pairs = opts->num_qps, + .socket_id = socket_id }; struct rte_cryptodev_qp_conf qp_conf = { @@ -158,14 +158,16 @@ cperf_initialize_cryptodev(struct cperf_options *opts, uint8_t *enabled_cdevs, return -EINVAL; } - ret = rte_cryptodev_queue_pair_setup(cdev_id, 0, + for (j = 0; j < opts->num_qps; j++) { + ret = rte_cryptodev_queue_pair_setup(cdev_id, j, &qp_conf, socket_id, session_pool_socket[socket_id]); if (ret < 0) { printf("Failed to setup queue pair %u on " - "cryptodev %u", 0, cdev_id); + "cryptodev %u", j, cdev_id); return -EINVAL; } + } ret = rte_cryptodev_start(cdev_id); if (ret < 0) { @@ -464,23 +466,29 @@ main(int argc, char **argv) if (!opts.silent) show_test_vector(t_vec); + uint16_t total_num_qps = nb_cryptodevs * opts.num_qps; + i = 0; + uint8_t qp_id = 0, cdev_index = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { - if (i == nb_cryptodevs) + if (i == total_num_qps) break; - cdev_id = enabled_cdevs[i]; + cdev_id = enabled_cdevs[cdev_index]; uint8_t socket_id = rte_cryptodev_socket_id(cdev_id); - ctx[cdev_id] = cperf_testmap[opts.test].constructor( - session_pool_socket[socket_id], cdev_id, 0, + ctx[i] = cperf_testmap[opts.test].constructor( + session_pool_socket[socket_id], cdev_id, qp_id, &opts, t_vec, &op_fns); - if (ctx[cdev_id] == NULL) { + if (ctx[i] == NULL) { RTE_LOG(ERR, USER1, "Test run constructor failed\n"); goto err; } + qp_id = (qp_id + 1) % opts.num_qps; + if (qp_id == 0) + cdev_index++; i++; } @@ -494,19 +502,17 @@ main(int argc, char **argv) i = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { - if (i == nb_cryptodevs) + if (i == total_num_qps) break; - cdev_id = enabled_cdevs[i]; - rte_eal_remote_launch(cperf_testmap[opts.test].runner, - ctx[cdev_id], lcore_id); + ctx[i], lcore_id); i++; } i = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { - if (i == nb_cryptodevs) + if (i == total_num_qps) break; rte_eal_wait_lcore(lcore_id); i++; @@ -525,15 +531,17 @@ main(int argc, char **argv) i = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { - if (i == nb_cryptodevs) + if (i == total_num_qps) break; - cdev_id = enabled_cdevs[i]; - - cperf_testmap[opts.test].destructor(ctx[cdev_id]); + cperf_testmap[opts.test].destructor(ctx[i]); i++; } + for (i = 0; i < nb_cryptodevs && + i < RTE_CRYPTO_MAX_DEVS; i++) + rte_cryptodev_stop(enabled_cdevs[i]); + free_test_vector(t_vec, &opts); printf("\n"); @@ -542,16 +550,20 @@ main(int argc, char **argv) err: i = 0; RTE_LCORE_FOREACH_SLAVE(lcore_id) { - if (i == nb_cryptodevs) + if (i == total_num_qps) break; cdev_id = enabled_cdevs[i]; - if (ctx[cdev_id] && cperf_testmap[opts.test].destructor) - cperf_testmap[opts.test].destructor(ctx[cdev_id]); + if (ctx[i] && cperf_testmap[opts.test].destructor) + cperf_testmap[opts.test].destructor(ctx[i]); i++; } + for (i = 0; i < nb_cryptodevs && + i < RTE_CRYPTO_MAX_DEVS; i++) + rte_cryptodev_stop(enabled_cdevs[i]); + free_test_vector(t_vec, &opts); printf("\n"); diff --git a/doc/guides/tools/cryptoperf.rst b/doc/guides/tools/cryptoperf.rst index 23b2b98..2fb0c66 100644 --- a/doc/guides/tools/cryptoperf.rst +++ b/doc/guides/tools/cryptoperf.rst @@ -192,6 +192,11 @@ The following are the appication command-line options: crypto_armv8 crypto_scheduler +* ``--qps `` + + Set the number of queue pairs per device (1 by default). + + * ``--optype `` Set operation type, where ``name`` is one of the following::