[v5] build: use platform option for generic and native

  The current meson option 'machine' should only specify the ISA, which is
not sufficient for Arm, where setting ISA implies other setting as well.
Use the existing 'platform' meson option to differentiate the type of
the build (native/generic) and set machine accordingly, unless the user
chooses to override it.
The 'machine' option also doesn't describe very well what it sets, so
introduce a new option 'cpu_instruction_set', but keep 'machine' for
backwards compatibility.
'machine' was setting the ISA in x86 builds and setting native/default
'build type' in aarch64 builds. These two new variables, taken
together, are now properly setting both for all architectures in a
uniform manner.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
---
 config/arm/meson.build              |  6 ++--
 config/meson.build                  | 53 ++++++++++++++++++++---------
 devtools/test-meson-builds.sh       |  9 ++---
 doc/guides/linux_gsg/build_dpdk.rst | 28 ++++++++++++++-
 meson_options.txt                   | 10 +++---
 5 files changed, 78 insertions(+), 28 deletions(-)
  

> -----Original Message-----
> From: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Sent: Tuesday, April 20, 2021 10:08 AM
> To: thomas@monjalon.net; david.marchand@redhat.com;
> bruce.richardson@intel.com; Honnappa.Nagarahalli@arm.com;
> Ruifeng.Wang@arm.com
> Cc: dev@dpdk.org; Juraj Linkeš <juraj.linkes@pantheon.tech>
> Subject: [PATCH v5] build: use platform option for generic and native
> 
> The current meson option 'machine' should only specify the ISA, which is not
> sufficient for Arm, where setting ISA implies other setting as well.
> Use the existing 'platform' meson option to differentiate the type of the build
> (native/generic) and set machine accordingly, unless the user chooses to
> override it.
> The 'machine' option also doesn't describe very well what it sets, so introduce a
> new option 'cpu_instruction_set', but keep 'machine' for backwards
> compatibility.
> 'machine' was setting the ISA in x86 builds and setting native/default 'build type'
> in aarch64 builds. These two new variables, taken together, are now properly
> setting both for all architectures in a uniform manner.
> 
> Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
> ---
>  config/arm/meson.build              |  6 ++--
>  config/meson.build                  | 53 ++++++++++++++++++++---------
>  devtools/test-meson-builds.sh       |  9 ++---
>  doc/guides/linux_gsg/build_dpdk.rst | 28 ++++++++++++++-
>  meson_options.txt                   | 10 +++---
>  5 files changed, 78 insertions(+), 28 deletions(-)
> 
> diff --git a/config/arm/meson.build b/config/arm/meson.build index
> 86cdb9b53b..59c788f1fa 100644
> --- a/config/arm/meson.build
> +++ b/config/arm/meson.build
> @@ -347,7 +347,7 @@ else
>  	soc = get_option('platform')
>  	soc_config = {}
>  	if not meson.is_cross_build()
> -		if machine == 'generic'
> +		if platform == 'generic'
>  			# generic build
>  			if soc != ''
>  				error('Building for a particular platform is ' +
> @@ -402,7 +402,7 @@ else
>  	else
>  		error('Unsupported Arm implementer: @0@.
> '.format(implementer_id) +
>  		      'Please add support for it or use the generic ' +
> -		      '(-Dmachine=generic) build.')
> +		      '(-Dplatform=generic) build.')
>  	endif
> 
>  	message('Arm implementer: ' + implementer_config['description'])
> @@ -417,7 +417,7 @@ else
>  		error('Unsupported part number @0@ of implementer @1@. '
>  		      .format(part_number, implementer_id) +
>  		      'Please add support for it or use the generic ' +
> -		      '(-Dmachine=generic) build.')
> +		      '(-Dplatform=generic) build.')
>  	endif
> 
>  	# add/overwrite flags in the proper order diff --git a/config/meson.build
> b/config/meson.build index 6e6ef8c0e1..1b6dcab8f5 100644
> --- a/config/meson.build
> +++ b/config/meson.build
> @@ -67,43 +67,64 @@ endif
>  disable_drivers = ''
>  enable_drivers = ''
> 
> -# set the machine type and cflags for it
> +platform = get_option('platform')
> +
> +# set the cpu_instruction_set and cflags for it
>  if meson.is_cross_build()
> -	machine = host_machine.cpu()
> +	cpu_instruction_set = host_machine.cpu()
>  else
> -	machine = get_option('machine')
> +	cpu_instruction_set = get_option('cpu_instruction_set')
> +	if get_option('machine') != 'auto'
> +		warning('The "machine" option is deprecated. ' +
> +		        'Please use "cpu_instruction_set" instead.')
> +		if cpu_instruction_set != 'auto'
> +			error('Setting both "machine" and ' +
> +			      '"cpu_instruction_set" is unsupported.')
> +		endif
> +		cpu_instruction_set = get_option('machine')
> +	endif
> +endif
> +
> +if platform == 'native'
> +	if cpu_instruction_set == 'auto'
> +		cpu_instruction_set = 'native'
> +	endif
> +elif platform == 'generic'
> +	if cpu_instruction_set == 'auto'
> +		cpu_instruction_set = 'generic'
> +	endif
>  endif
> 
> -# machine type 'generic' is special, it selects the per arch agreed common -#
> minimal baseline needed for DPDK. Machine type 'default' is also supported -#
> with the same meaning for backwards compatibility.
> +# cpu_instruction_set 'generic' is special, it selects the per arch
> +agreed # common minimal baseline needed for DPDK. cpu_instruction_set
> +'default' is # also supported with the same meaning for backwards
> compatibility.
>  # That might not be the most optimized, but the most portable version while  #
> still being able to support the CPU features required for DPDK.
>  # This can be bumped up by the DPDK project, but it can never be an  # invariant
> like 'native'
> -if machine == 'default' or machine == 'generic'
> +if cpu_instruction_set == 'generic' or cpu_instruction_set == 'default'
>  	if host_machine.cpu_family().startswith('x86')
> -		# matches the old pre-meson build systems generic machine
> -		machine = 'corei7'
> +		# matches the old pre-meson build systems generic
> cpu_instruction_set
> +		cpu_instruction_set = 'corei7'
>  	elif host_machine.cpu_family().startswith('arm')
> -		machine = 'armv7-a'
> +		cpu_instruction_set = 'armv7-a'
>  	elif host_machine.cpu_family().startswith('aarch')
>  		# arm64 manages generic config in config/arm/meson.build
> -		machine = 'generic'
> +		cpu_instruction_set = 'generic'
>  	elif host_machine.cpu_family().startswith('ppc')
> -		machine = 'power8'
> +		cpu_instruction_set = 'power8'
>  	endif
>  endif
> 
> -dpdk_conf.set('RTE_MACHINE', machine)
> +dpdk_conf.set('RTE_MACHINE', cpu_instruction_set)
>  machine_args = []
> 
>  # ppc64 does not support -march= at all, use -mcpu and -mtune for that  if
> host_machine.cpu_family().startswith('ppc')
> -	machine_args += '-mcpu=' + machine
> -	machine_args += '-mtune=' + machine
> +	machine_args += '-mcpu=' + cpu_instruction_set
> +	machine_args += '-mtune=' + cpu_instruction_set
>  else
> -	machine_args += '-march=' + machine
> +	machine_args += '-march=' + cpu_instruction_set
>  endif
> 
>  toolchain = cc.get_id()
> diff --git a/devtools/test-meson-builds.sh b/devtools/test-meson-builds.sh index
> daf817ac3e..7edbd6c717 100755
> --- a/devtools/test-meson-builds.sh
> +++ b/devtools/test-meson-builds.sh
> @@ -223,12 +223,13 @@ done
>  # test compilation with minimal x86 instruction set  # Set the install path for
> libraries to "lib" explicitly to prevent problems  # with pkg-config prefixes if
> installed in "lib/x86_64-linux-gnu" later.
> -generic_machine='nehalem'
> -if ! check_cc_flags "-march=$generic_machine" ; then
> -	generic_machine='corei7'
> +generic_isa='nehalem'
> +if ! check_cc_flags "-march=$generic_isa" ; then
> +	generic_isa='corei7'
>  fi
>  build build-x86-generic cc skipABI -Dcheck_includes=true \
> -	-Dlibdir=lib -Dmachine=$generic_machine $use_shared
> +	-Dlibdir=lib -Dcpu_instruction_set=$generic_isa \
> +	$use_shared
> 
>  # 32-bit with default compiler
>  if check_cc_flags '-m32' ; then
> diff --git a/doc/guides/linux_gsg/build_dpdk.rst
> b/doc/guides/linux_gsg/build_dpdk.rst
> index f78eef2517..787b2c2d1f 100644
> --- a/doc/guides/linux_gsg/build_dpdk.rst
> +++ b/doc/guides/linux_gsg/build_dpdk.rst
> @@ -89,7 +89,33 @@ to a regular "debug" build, you can either:
>  * run ``meson configure -Dbuildtype=debug`` inside the build folder after the
> initial meson run.
> 
>  Other options are specific to the DPDK project but can be adjusted similarly.
> -To set the "max_lcores" value to 256, for example, you can either:
> +The "platform" option specifies a set a configuration parameters that
> +will be used. The valid values are:
> +
> +* ``-Dplatform=native`` will tailor the configuration to the build machine.
> +
> +* ``-Dplatform=generic`` will use configuration that works on all
> +machines of the same architecture as the build machine.
> +
> +* ``-Dplatform=<Arm_SoC>`` will use configuration optimized for a
> +particular Arm SoC. Consult the "socs" dictionary in
> +config/arm/meson.build to see which SoC are supported.
> +
> +An important configuration parameter that "platform" sets is the
> +instruction set to use in x86 and ppc builds:
> +
> +* ``-Dplatform=native`` sets "cpu_instruction_set" to "native", which
> +configures -march, -mcpu, -mtune to "native".
> +
> +* ``-Dplatform=generic`` sets "cpu_instruction_set" to "generic", which
> +configures -march, -mcpu, -mtune to a common minimal baseline needed for
> DPDK.
> +
> +"cpu_instruction_set" is not used in Arm builds, as setting the
> +instruction set without other parameters leads to inferior builds. The
> +way to tailor Arm builds is to build for an SoC or a native build using the
> "platform" option.
> +
> +The values determined by the "platform" parameter may be overwritten.
> +For example, to set the "max_lcores" value to 256, you can either:
> 

I documented the new options in more detail here. I don't think this is the right place to document these, since this is only the Linux guide and the options should be os agnostic. I'm not sure where the better place to put this would be.

>  * pass ``-Dmax_lcores=256`` to meson when configuring the build folder initially
> 
> diff --git a/meson_options.txt b/meson_options.txt index
> b78f3bd9d5..07db5146db 100644
> --- a/meson_options.txt
> +++ b/meson_options.txt
> @@ -2,6 +2,8 @@
> 
>  option('check_includes', type: 'boolean', value: false,
>  	description: 'build "chkincs" to verify each header file can compile
> alone')
> +option('cpu_instruction_set', type: 'string', value: 'auto',
> +	description: 'Set the target machine ISA (instruction set
> +architecture). Will be set according to the platform option by
> +default.')
>  option('developer_mode', type: 'feature',
>  	description: 'turn on additional build checks relevant for DPDK
> developers')  option('disable_drivers', type: 'string', value: '', @@ -26,16 +28,16
> @@ option('include_subdir_arch', type: 'string', value: '',
>  	description: 'subdirectory where to install arch-dependent headers')
> option('kernel_dir', type: 'string', value: '',
>  	description: 'Path to the kernel for building kernel modules. Headers
> must be in $kernel_dir or $kernel_dir/build. Modules will be installed in
> /lib/modules.') -option('machine', type: 'string', value: 'native',
> -	description: 'set the target machine type or "generic", a build usable on
> all machines of the build machine architecture or "native", which lets the
> compiler pick the architecture of the build machine.')
> +option('machine', type: 'string', value: 'auto',
> +	description: 'Alias of cpu_instruction_set.')
>  option('max_ethports', type: 'integer', value: 32,
>  	description: 'maximum number of Ethernet devices')
> option('max_lcores', type: 'integer', value: 128,
>  	description: 'maximum number of cores/threads supported by EAL')
> option('max_numa_nodes', type: 'integer', value: 32,
>  	description: 'maximum number of NUMA nodes supported by EAL') -
> option('platform', type: 'string', value: '',
> -	description: 'use configuration for a particular platform (such as a SoC).')
> +option('platform', type: 'string', value: 'generic',
> +	description: 'Platform to build, either "native", "generic" or an Arm
> +SoC. Please refer to the Linux build guide for more information.')

The other thing we need to discuss is the fact that we're changing the behavior in this patch from machine=native to platform=generic (and the implied cpu_instruction_set=genetic which equals to machine=generic). We need more feedback on this - Thomas, David? Should we include more people? The reason for this change is we want the build without any meson options (specified on cmdline) to be usable on as many different machines of the same architecture (this is my understanding form the discussion with Bruce), which is the generic platform. We could preserve the current behavior by setting the default to native, so we just need more discussion.

>  option('enable_trace_fp', type: 'boolean', value: false,
>  	description: 'enable fast path trace points.')  option('tests', type:
> 'boolean', value: true,
> --
> 2.20.1
  

20/04/2021 10:16, Juraj Linkeš:
> > option('platform', type: 'string', value: '',
> > -	description: 'use configuration for a particular platform (such as a SoC).')
> > +option('platform', type: 'string', value: 'generic',
> > +	description: 'Platform to build, either "native", "generic" or an Arm
> > +SoC. Please refer to the Linux build guide for more information.')
> 
> The other thing we need to discuss is the fact that we're changing the behavior in this patch from machine=native to platform=generic (and the implied cpu_instruction_set=genetic which equals to machine=generic). We need more feedback on this - Thomas, David? Should we include more people? The reason for this change is we want the build without any meson options (specified on cmdline) to be usable on as many different machines of the same architecture (this is my understanding form the discussion with Bruce), which is the generic platform. We could preserve the current behavior by setting the default to native, so we just need more discussion.

I don't understand why you need to change the behaviour.
If we change the behaviour, it must be done carefully
after a deprecation notice and an agreement of the Technical Board.
  

> -----Original Message-----
> From: Thomas Monjalon <thomas@monjalon.net>
> Sent: Tuesday, April 20, 2021 10:36 AM
> To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> Cc: david.marchand@redhat.com; bruce.richardson@intel.com;
> Honnappa.Nagarahalli@arm.com; Ruifeng.Wang@arm.com; dev@dpdk.org
> Subject: Re: [PATCH v5] build: use platform option for generic and native
> 
> 20/04/2021 10:16, Juraj Linkeš:
> > > option('platform', type: 'string', value: '',
> > > -	description: 'use configuration for a particular platform (such as a SoC).')
> > > +option('platform', type: 'string', value: 'generic',
> > > +	description: 'Platform to build, either "native", "generic" or an
> > > +Arm SoC. Please refer to the Linux build guide for more
> > > +information.')
> >
> > The other thing we need to discuss is the fact that we're changing the behavior
> in this patch from machine=native to platform=generic (and the implied
> cpu_instruction_set=genetic which equals to machine=generic). We need more
> feedback on this - Thomas, David? Should we include more people? The reason
> for this change is we want the build without any meson options (specified on
> cmdline) to be usable on as many different machines of the same architecture
> (this is my understanding form the discussion with Bruce), which is the generic
> platform. We could preserve the current behavior by setting the default to
> native, so we just need more discussion.
> 
> I don't understand why you need to change the behaviour.

Well, we don't *need* to change the behavior - I don't mind it either way. We've come to this point after discussing the change with Bruce. I believe Bruce mentioned the intention of the build without any meson cmdline options was to produce a build usable on most machines that match the build architecture and the generic build fits better (native build being more tailored to the build machine as opposed to the generic build).

I don't want to speak for Bruce though, so it'd be the best if he gave his thoughts on the matter.

> If we change the behaviour, it must be done carefully after a deprecation notice
> and an agreement of the Technical Board.
> 
> 

Ok, one thing we could do is preserve the behavior in this patch and go through this process with a separate change.
  

Hi Linkeš，

This patch cause vhost/virtio basic pvp test with 1518 packet size performance drop about 9% on IA platform.

BR,
Yinan

> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Juraj Linkeš
> Sent: 2021年4月21日 16:38
> To: Thomas Monjalon <thomas@monjalon.net>
> Cc: david.marchand@redhat.com; Richardson, Bruce
> <bruce.richardson@intel.com>; Honnappa.Nagarahalli@arm.com;
> Ruifeng.Wang@arm.com; dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v5] build: use platform option for generic
> and native
> 
> 
> 
> > -----Original Message-----
> > From: Thomas Monjalon <thomas@monjalon.net>
> > Sent: Tuesday, April 20, 2021 10:36 AM
> > To: Juraj Linkeš <juraj.linkes@pantheon.tech>
> > Cc: david.marchand@redhat.com; bruce.richardson@intel.com;
> > Honnappa.Nagarahalli@arm.com; Ruifeng.Wang@arm.com;
> dev@dpdk.org
> > Subject: Re: [PATCH v5] build: use platform option for generic and native
> >
> > 20/04/2021 10:16, Juraj Linkeš:
> > > > option('platform', type: 'string', value: '',
> > > > -	description: 'use configuration for a particular platform (such as a
> SoC).')
> > > > +option('platform', type: 'string', value: 'generic',
> > > > +	description: 'Platform to build, either "native", "generic" or an
> > > > +Arm SoC. Please refer to the Linux build guide for more
> > > > +information.')
> > >
> > > The other thing we need to discuss is the fact that we're changing the
> behavior
> > in this patch from machine=native to platform=generic (and the implied
> > cpu_instruction_set=genetic which equals to machine=generic). We need
> more
> > feedback on this - Thomas, David? Should we include more people? The
> reason
> > for this change is we want the build without any meson options (specified
> on
> > cmdline) to be usable on as many different machines of the same
> architecture
> > (this is my understanding form the discussion with Bruce), which is the
> generic
> > platform. We could preserve the current behavior by setting the default
> to
> > native, so we just need more discussion.
> >
> > I don't understand why you need to change the behaviour.
> 
> Well, we don't *need* to change the behavior - I don't mind it either way.
> We've come to this point after discussing the change with Bruce. I believe
> Bruce mentioned the intention of the build without any meson cmdline
> options was to produce a build usable on most machines that match the
> build architecture and the generic build fits better (native build being more
> tailored to the build machine as opposed to the generic build).
> 
> I don't want to speak for Bruce though, so it'd be the best if he gave his
> thoughts on the matter.
> 
> > If we change the behaviour, it must be done carefully after a deprecation
> notice
> > and an agreement of the Technical Board.
> >
> >
> 
> Ok, one thing we could do is preserve the behavior in this patch and go
> through this process with a separate change.