[dpdk-dev,2/2] meson: add tests app to build

  This patch enables the test/test app to be built. It also adds
the test binary to be a meson-test, which allows the meson test
infrastructure to be used to run tests.

Tests are listed using the same test binary, however each test
sets a different DPDK_TEST environment variable. The string contents
of this DPDK_TEST env var is entered in the command line interface.
As such, the familiar test names such as "ring_perf_autotest" etc
are valid tests to run using this meson test infrastructure.

Note that the tests are run serially, given that we cannot run
multiple primary processes at a time. As each test must initialize
EAL this takes some time depending on the number of hugepages.
In future, we could improve this to run multiple tests from one
EAL init, but it is out of scope for this patchset.

Finally, an option to build the tests is added to the meson build
options. When disabled, the unit test code in test/test is not
compiled. The default is set to 'true'. To disable, run:

$ meson configure -Dtests=false

Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
---
 meson.build           |   1 +
 meson_options.txt     |   2 +
 test/meson.build      |  32 +++++++
 test/test/meson.build | 248 ++++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 283 insertions(+)
 create mode 100644 test/meson.build
 create mode 100644 test/test/meson.build
  

On Mon, Dec 18, 2017 at 11:53:58AM +0000, Harry van Haaren wrote:
> This patch enables the test/test app to be built. It also adds
> the test binary to be a meson-test, which allows the meson test
> infrastructure to be used to run tests.
> 
> Tests are listed using the same test binary, however each test
> sets a different DPDK_TEST environment variable. The string contents
> of this DPDK_TEST env var is entered in the command line interface.
> As such, the familiar test names such as "ring_perf_autotest" etc
> are valid tests to run using this meson test infrastructure.
> 
> Note that the tests are run serially, given that we cannot run
> multiple primary processes at a time. As each test must initialize
> EAL this takes some time depending on the number of hugepages.
> In future, we could improve this to run multiple tests from one
> EAL init, but it is out of scope for this patchset.
> 
> Finally, an option to build the tests is added to the meson build
> options. When disabled, the unit test code in test/test is not
> compiled. The default is set to 'true'. To disable, run:
> 
> $ meson configure -Dtests=false
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---

<snip>
> +
> +test_names = [
> +	'acl_autotest',
> +	'alarm_autotest',
> +	'atomic_autotest',
> +	'byteorder_autotest',
> +	'cmdline_autotest',
> +	'common_autotest',
> +	'cpuflags_autotest',
> +	'crc_autotest',
> +	'cycles_autotest',
> +	'debug_autotest',
> +	'devargs_autotest',
> +	'distributor_autotest',
> +	'distributor_perf_autotest',
> +	'eal_flags_autotest',
> +	'eal_fs_autotest',
> +	'efd_autotest',
> +	'efd_perf_autotest',
> +	'errno_autotest',
> +	'event_ring_autotest',
> +	'eventdev_common_autotest',
> +	'eventdev_octeontx_autotest',
> +	'eventdev_sw_autotest',
> +	'func_reentrancy_autotest',
> +	'has_scaling_autotest',
> +	'hash_autotest',
> +	'hash_functions_autotest',
> +	'hash_multiwriter_autotest',
> +	'hash_perf_autotest',
> +	'interrupt_autotest',
> +	'kni_autotest',
> +	'kvargs_autotest',
> +	'logs_autotest',
> +	'lpm6_autotest',
> +	'lpm6_perf_autotest',
> +	'lpm_autotest',
> +	'lpm_perf_autotest',
> +	'malloc_autotest',
> +	'mbuf_autotest',
> +	'memcpy_autotest',
> +	'memcpy_perf_autotest',
> +	'memory_autotest',
> +	'mempool_autotest',
> +	'mempool_perf_autotest',
> +	'memzone_autotest',
> +	'meter_autotest',
> +	'multiprocess_autotest',
> +	'per_lcore_autotest',
> +	'pmd_perf_autotest',
> +	'power_acpi_cpufreq_autotest',
> +	'power_autotest',
> +	'power_kvm_vm_autotest',
> +	'prefetch_autotest',
> +	'red_all',
> +	'red_autotest',
> +	'red_perf',
> +	'reorder_autotest',
> +	'ring_autotest',
> +	'ring_perf_autotest',
> +	'rwlock_autotest',
> +	'sched_autotest',
> +	'service_autotest',
> +	'spinlock_autotest',
> +	'table_autotest',
> +	'tailq_autotest',
> +	'thash_autotest',
> +	'timer_autotest',
> +	'timer_perf__autotest',
> +	'timer_racecond_autotest',
> +	'user_delay_us',
> +	'version_autotest',
> +]

I think we should find a way to remove this long list of test names in
the build file. The list of available tests is built up dynamically
inside the test binary, so I'm wondering if we can add to the binary an
"all" option to run each autotest straight after each other?

> +
> +if dpdk_conf.has('RTE_LIBRTE_PDUMP')
> +	test_deps += 'pdump'
> +endif
> +if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
> +	test_deps += 'pmd_i40e'
> +endif
> +if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
> +	test_deps += 'pmd_ixgbe'
> +endif
> +

This should be moved up beside the other dependencies

> +test_dep_objs = []
> +foreach d:test_deps
> +	def_lib = get_option('default_library')
> +	test_dep_objs += get_variable(def_lib + '_rte_' + d)
> +endforeach
> +
> +link_libs = []
> +if get_option('default_library') == 'static'
> +	link_libs = dpdk_drivers
> +endif
> +

A note for the future: since much of this looks similar to that in
testpmd we should look to see if we can find a way to easily consolidate
these build commands inside a foreach loop like is done for the libs,
drivers and examples.

> +if get_option('tests')
> +	dpdk_test = executable('dpdk-test',
> +		test_sources,
> +		link_whole: link_libs,
> +		dependencies: test_dep_objs,
> +		install_rpath: driver_install_path,
> +		install: false)
> +
> +	# some perf tests (eg: memcpy perf autotest)take very long
> +	# to complete, so timeout to 10 minutes
> +	timeout_seconds = 600
> +
> +	foreach t:test_names
> +		test(t, dpdk_test,
> +			env : ['DPDK_TEST='+t],
> +			timeout : timeout_seconds,
> +			is_parallel : false)
> +	endforeach
> +endif
> -- 
> 2.7.4
>
  

> From: Richardson, Bruce
> Sent: Monday, December 18, 2017 1:55 PM
> To: Van Haaren, Harry <harry.van.haaren@intel.com>
> Cc: dev@dpdk.org
> Subject: Re: [PATCH 2/2] meson: add tests app to build
> 
> On Mon, Dec 18, 2017 at 11:53:58AM +0000, Harry van Haaren wrote:
> > This patch enables the test/test app to be built. It also adds
> > the test binary to be a meson-test, which allows the meson test
> > infrastructure to be used to run tests.
> >
> > Tests are listed using the same test binary, however each test
> > sets a different DPDK_TEST environment variable. The string contents
> > of this DPDK_TEST env var is entered in the command line interface.
> > As such, the familiar test names such as "ring_perf_autotest" etc
> > are valid tests to run using this meson test infrastructure.
> >
> > Note that the tests are run serially, given that we cannot run
> > multiple primary processes at a time. As each test must initialize
> > EAL this takes some time depending on the number of hugepages.
> > In future, we could improve this to run multiple tests from one
> > EAL init, but it is out of scope for this patchset.
> >
> > Finally, an option to build the tests is added to the meson build
> > options. When disabled, the unit test code in test/test is not
> > compiled. The default is set to 'true'. To disable, run:
> >
> > $ meson configure -Dtests=false
> >
> > Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> > ---
> 
> <snip>
> > +
> > +test_names = [
> > +	'acl_autotest',
> > +	'alarm_autotest',
<snip>
> > +	'user_delay_us',
> > +	'version_autotest',
> > +]
> 
> I think we should find a way to remove this long list of test names in
> the build file. The list of available tests is built up dynamically
> inside the test binary, so I'm wondering if we can add to the binary an
> "all" option to run each autotest straight after each other?


Agree - it would be nice if we don't have to specify tests here. We could add a flag the test binary, to output the tests that it contains? Then its about having Meson read this file before running tests, to "gather" the tests available..

There's options, but they will probably all be more complex than just having a list of tests.

Re Run all tests: Currently   $ meson test     will run all unit tests.


> > +
> > +if dpdk_conf.has('RTE_LIBRTE_PDUMP')
> > +	test_deps += 'pdump'
> > +endif
> > +if dpdk_conf.has('RTE_LIBRTE_I40E_PMD')
> > +	test_deps += 'pmd_i40e'
> > +endif
> > +if dpdk_conf.has('RTE_LIBRTE_IXGBE_PMD')
> > +	test_deps += 'pmd_ixgbe'
> > +endif
> > +
> 
> This should be moved up beside the other dependencies
> 
> > +test_dep_objs = []
> > +foreach d:test_deps
> > +	def_lib = get_option('default_library')
> > +	test_dep_objs += get_variable(def_lib + '_rte_' + d)
> > +endforeach
> > +
> > +link_libs = []
> > +if get_option('default_library') == 'static'
> > +	link_libs = dpdk_drivers
> > +endif
> > +
> 
> A note for the future: since much of this looks similar to that in
> testpmd we should look to see if we can find a way to easily consolidate
> these build commands inside a foreach loop like is done for the libs,
> drivers and examples.

+1, good idea for a cleanup later.

Meson >= 0.44 has a "disabler" concept[1], which is a clean way to disable components in the build if dependencies aren't found. Eg: specific tests or sample apps, can be disabled from the build without the if() logic required as below.

[1] http://mesonbuild.com/Disabler.html


> > +if get_option('tests')
> > +	dpdk_test = executable('dpdk-test',
> > +		test_sources,
> > +		link_whole: link_libs,
> > +		dependencies: test_dep_objs,
> > +		install_rpath: driver_install_path,
> > +		install: false)
> > +
> > +	# some perf tests (eg: memcpy perf autotest)take very long
> > +	# to complete, so timeout to 10 minutes
> > +	timeout_seconds = 600
> > +
> > +	foreach t:test_names
> > +		test(t, dpdk_test,
> > +			env : ['DPDK_TEST='+t],
> > +			timeout : timeout_seconds,
> > +			is_parallel : false)
> > +	endforeach
> > +endif
> > --
> > 2.7.4
> >
  

On Mon, Dec 18, 2017 at 11:53:58AM +0000, Harry van Haaren wrote:
> This patch enables the test/test app to be built. It also adds
> the test binary to be a meson-test, which allows the meson test
> infrastructure to be used to run tests.
> 
> Tests are listed using the same test binary, however each test
> sets a different DPDK_TEST environment variable. The string contents
> of this DPDK_TEST env var is entered in the command line interface.
> As such, the familiar test names such as "ring_perf_autotest" etc
> are valid tests to run using this meson test infrastructure.
> 
> Note that the tests are run serially, given that we cannot run
> multiple primary processes at a time. As each test must initialize
> EAL this takes some time depending on the number of hugepages.
> In future, we could improve this to run multiple tests from one
> EAL init, but it is out of scope for this patchset.
> 
> Finally, an option to build the tests is added to the meson build
> options. When disabled, the unit test code in test/test is not
> compiled. The default is set to 'true'. To disable, run:
> 
> $ meson configure -Dtests=false
> 
> Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
> ---
>  meson.build           |   1 +
>  meson_options.txt     |   2 +
>  test/meson.build      |  32 +++++++
>  test/test/meson.build | 248 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 283 insertions(+)
>  create mode 100644 test/meson.build
>  create mode 100644 test/test/meson.build
> 
> diff --git a/meson.build b/meson.build
> index 04eea72..9132594 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -60,6 +60,7 @@ subdir('drivers')
>  # build binaries and installable tools
>  subdir('usertools')
>  subdir('app')
> +subdir('test')
>  
>  # build any examples explicitly requested - useful for developers
>  if get_option('examples') != ''
> diff --git a/meson_options.txt b/meson_options.txt
> index f2558fe..1a674aa 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -18,3 +18,5 @@ option('per_library_versions', type: 'boolean', value: true,
>  	description: 'true: each lib gets its own version number, false: DPDK version used for each lib')
>  option('use_hpet', type: 'boolean', value: false,
>  	description: 'use HPET timer in EAL')
> +option('tests', type: 'boolean', value: true,
> +	description: 'build unit tests')
> diff --git a/test/meson.build b/test/meson.build
> new file mode 100644
> index 0000000..4250eb7
> --- /dev/null
> +++ b/test/meson.build
> @@ -0,0 +1,32 @@
> +#   BSD LICENSE
> +#
> +#   Copyright(c) 2017 Intel Corporation. All rights reserved.
> +#   All rights reserved.
> +#
> +#   Redistribution and use in source and binary forms, with or without
> +#   modification, are permitted provided that the following conditions
> +#   are met:
> +#
> +#     * Redistributions of source code must retain the above copyright
> +#       notice, this list of conditions and the following disclaimer.
> +#     * Redistributions in binary form must reproduce the above copyright
> +#       notice, this list of conditions and the following disclaimer in
> +#       the documentation and/or other materials provided with the
> +#       distribution.
> +#     * Neither the name of Intel Corporation nor the names of its
> +#       contributors may be used to endorse or promote products derived
> +#       from this software without specific prior written permission.
> +#
> +#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +subdir('test')

Can you do a V2 of this set with the SPDX tag in place of the license
text. Otherwise, the set looks good enough to me.

/Bruce