[v2] devtools: better freebsd support

  - As "readlink -e" and "readlink -m" do not exist on freebsd,
  use "readlink -f", it should not have any impact in these cases.
- "sed -ri" is invalid on freebsd and should be replaced by
  "sed -ri=''"
- Use gmake instead of make.

This fixes the following command:
  SYSDIR=/usr/src/sys ./devtools/test-build.sh \
    -j4 x86_64-native-freebsd-gcc

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
---

v2:
- remove sed_ri() function and use 'sed -ri=""' as suggested by Bruce

 devtools/check-dup-includes.sh |  2 +-
 devtools/checkpatches.sh       |  8 ++--
 devtools/get-maintainer.sh     |  2 +-
 devtools/load-devel-config     |  4 +-
 devtools/test-build.sh         | 94 ++++++++++++++++++++++--------------------
 devtools/validate-abi.sh       |  2 +-
 6 files changed, 58 insertions(+), 54 deletions(-)
  

05/07/2019 15:58, Olivier Matz:
> - As "readlink -e" and "readlink -m" do not exist on freebsd,
>   use "readlink -f", it should not have any impact in these cases.
> - "sed -ri" is invalid on freebsd and should be replaced by
>   "sed -ri=''"
> - Use gmake instead of make.
> 
> This fixes the following command:
>   SYSDIR=/usr/src/sys ./devtools/test-build.sh \
>     -j4 x86_64-native-freebsd-gcc
> 
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
> --- a/devtools/test-build.sh
> +++ b/devtools/test-build.sh
> +[ -z $MAKE ] && command -v gmake > /dev/null && MAKE=gmake
> +[ -z $MAKE ] && command -v make > /dev/null && MAKE=make
> +[ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1

I'm surprised it works, given the -e in this script.
I would be more confortable with "if/elif" constructs.
  

On 05/07/2019 14:58, Olivier Matz wrote:
> - As "readlink -e" and "readlink -m" do not exist on freebsd,
>    use "readlink -f", it should not have any impact in these cases.
> - "sed -ri" is invalid on freebsd and should be replaced by
>    "sed -ri=''"
> - Use gmake instead of make.
>
> This fixes the following command:
>    SYSDIR=/usr/src/sys ./devtools/test-build.sh \
>      -j4 x86_64-native-freebsd-gcc
>
> Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> ---
>
> v2:
> - remove sed_ri() function and use 'sed -ri=""' as suggested by Bruce
>
>   devtools/check-dup-includes.sh |  2 +-
>   devtools/checkpatches.sh       |  8 ++--
>   devtools/get-maintainer.sh     |  2 +-
>   devtools/load-devel-config     |  4 +-
>   devtools/test-build.sh         | 94 ++++++++++++++++++++++--------------------
>   devtools/validate-abi.sh       |  2 +-
>   6 files changed, 58 insertions(+), 54 deletions(-)
>
> diff --git a/devtools/check-dup-includes.sh b/devtools/check-dup-includes.sh
> index e4c2748c6..591599949 100755
> --- a/devtools/check-dup-includes.sh
> +++ b/devtools/check-dup-includes.sh
> @@ -5,7 +5,7 @@
>   # Check C files in git repository for duplicated includes.
>   # Usage: devtools/check-dup-includes.sh [directory]
>   
> -dir=${1:-$(dirname $(readlink -m $0))/..}
> +dir=${1:-$(dirname $(readlink -f $0))/..}
>   cd $dir
>   
>   # speed up by ignoring Unicode details
> diff --git a/devtools/checkpatches.sh b/devtools/checkpatches.sh
> index 560e6ce93..8e2beee16 100755
> --- a/devtools/checkpatches.sh
> +++ b/devtools/checkpatches.sh
> @@ -7,9 +7,9 @@
>   # - DPDK_CHECKPATCH_CODESPELL
>   # - DPDK_CHECKPATCH_LINE_LENGTH
>   # - DPDK_CHECKPATCH_OPTIONS
> -. $(dirname $(readlink -e $0))/load-devel-config
> +. $(dirname $(readlink -f $0))/load-devel-config
>   
> -VALIDATE_NEW_API=$(dirname $(readlink -e $0))/check-symbol-change.sh
> +VALIDATE_NEW_API=$(dirname $(readlink -f $0))/check-symbol-change.sh
>   
>   # Enable codespell by default. This can be overwritten from a config file.
>   # Codespell can also be enabled by setting DPDK_CHECKPATCH_CODESPELL to a valid path
> @@ -66,7 +66,7 @@ check_forbidden_additions() { # <patch>
>   		-v EXPRESSIONS="rte_panic\\\( rte_exit\\\(" \
>   		-v RET_ON_FAIL=1 \
>   		-v MESSAGE='Using rte_panic/rte_exit' \
> -		-f $(dirname $(readlink -e $0))/check-forbidden-tokens.awk \
> +		-f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \
>   		"$1" || res=1
>   
>   	# svg figures must be included with wildcard extension
> @@ -75,7 +75,7 @@ check_forbidden_additions() { # <patch>
>   		-v EXPRESSIONS='::[[:space:]]*[^[:space:]]*\\.svg' \
>   		-v RET_ON_FAIL=1 \
>   		-v MESSAGE='Using explicit .svg extension instead of .*' \
> -		-f $(dirname $(readlink -e $0))/check-forbidden-tokens.awk \
> +		-f $(dirname $(readlink -f $0))/check-forbidden-tokens.awk \
>   		"$1" || res=1
>   
>   	return $res
> diff --git a/devtools/get-maintainer.sh b/devtools/get-maintainer.sh
> index b9160486a..85740f5af 100755
> --- a/devtools/get-maintainer.sh
> +++ b/devtools/get-maintainer.sh
> @@ -5,7 +5,7 @@
>   
>   # Load config options:
>   # - DPDK_GETMAINTAINER_PATH
> -. $(dirname $(readlink -e $0))/load-devel-config
> +. $(dirname $(readlink -f $0))/load-devel-config
>   
>   options="--no-git-fallback"
>   options="$options --no-rolestats"
> diff --git a/devtools/load-devel-config b/devtools/load-devel-config
> index 4f43cb352..380c79db4 100644
> --- a/devtools/load-devel-config
> +++ b/devtools/load-devel-config
> @@ -6,7 +6,7 @@ test ! -r /etc/dpdk/devel.config ||
>   test ! -r ~/.config/dpdk/devel.config ||
>           . ~/.config/dpdk/devel.config
>   # from local file
> -test ! -r $(dirname $(readlink -m $0))/../.develconfig ||
> -        . $(dirname $(readlink -m $0))/../.develconfig
> +test ! -r $(dirname $(readlink -f $0))/../.develconfig ||
> +        . $(dirname $(readlink -f $0))/../.develconfig
>   
>   # The config files must export variables in the shell style
> diff --git a/devtools/test-build.sh b/devtools/test-build.sh
> index 9b50bf73d..22d409bb0 100755
> --- a/devtools/test-build.sh
> +++ b/devtools/test-build.sh
> @@ -28,7 +28,7 @@ default_path=$PATH
>   # - LIBSSO_SNOW3G_PATH
>   # - LIBSSO_KASUMI_PATH
>   # - LIBSSO_ZUC_PATH
> -. $(dirname $(readlink -e $0))/load-devel-config
> +. $(dirname $(readlink -f $0))/load-devel-config
>   
>   print_usage () {
>   	echo "usage: $(basename $0) [-h] [-jX] [-s] [config1 [config2] ...]]"
> @@ -57,6 +57,10 @@ print_help () {
>   	END_OF_HELP
>   }
>   
> +[ -z $MAKE ] && command -v gmake > /dev/null && MAKE=gmake
> +[ -z $MAKE ] && command -v make > /dev/null && MAKE=make
> +[ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1
> +
>   J=$DPDK_MAKE_JOBS
>   short=false
>   unset verbose
> @@ -90,7 +94,7 @@ trap "signal=INT ; trap - INT ; kill -INT $$" INT
>   # notify result on exit
>   trap on_exit EXIT
>   
> -cd $(dirname $(readlink -m $0))/..
> +cd $(dirname $(readlink -f $0))/..
>   
>   reset_env ()
>   {
> @@ -127,83 +131,83 @@ config () # <directory> <target> <options>
>   	fi
>   	if [ ! -e $1/.config ] || $reconfig ; then
>   		echo "================== Configure $1"
> -		make T=$2 O=$1 config
> +		${MAKE} T=$2 O=$1 config
>   
>   		echo 'Customize configuration'
>   		# Built-in options (lowercase)
>   		! echo $3 | grep -q '+default' || \
> -		sed -ri 's,(RTE_MACHINE=")native,\1default,' $1/.config
> +		sed -ri="" 's,(RTE_MACHINE=")native,\1default,' $1/.config
>   		echo $3 | grep -q '+next' || \
> -		sed -ri           's,(NEXT_ABI=)y,\1n,' $1/.config
> +		sed -ri=""           's,(NEXT_ABI=)y,\1n,' $1/.config
>   		! echo $3 | grep -q '+shared' || \
> -		sed -ri         's,(SHARED_LIB=)n,\1y,' $1/.config
> +		sed -ri=""         's,(SHARED_LIB=)n,\1y,' $1/.config
>   		! echo $3 | grep -q '+debug' || ( \
> -		sed -ri  's,(RTE_LOG_DP_LEVEL=).*,\1RTE_LOG_DEBUG,' $1/.config
> -		sed -ri           's,(_DEBUG.*=)n,\1y,' $1/.config
> -		sed -ri            's,(_STAT.*=)n,\1y,' $1/.config
> -		sed -ri 's,(TEST_PMD_RECORD_.*=)n,\1y,' $1/.config )
> +		sed -ri=""  's,(RTE_LOG_DP_LEVEL=).*,\1RTE_LOG_DEBUG,' $1/.config
> +		sed -ri=""           's,(_DEBUG.*=)n,\1y,' $1/.config
> +		sed -ri=""            's,(_STAT.*=)n,\1y,' $1/.config
> +		sed -ri="" 's,(TEST_PMD_RECORD_.*=)n,\1y,' $1/.config )
>   
>   		# Automatic configuration
>   		test "$DPDK_DEP_NUMA" != n || \
> -		sed -ri             's,(NUMA.*=)y,\1n,' $1/.config
> -		sed -ri    's,(LIBRTE_IEEE1588=)n,\1y,' $1/.config
> -		sed -ri             's,(BYPASS=)n,\1y,' $1/.config
> +		sed -ri=""             's,(NUMA.*=)y,\1n,' $1/.config
> +		sed -ri=""    's,(LIBRTE_IEEE1588=)n,\1y,' $1/.config
> +		sed -ri=""             's,(BYPASS=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_ARCHIVE" != y || \
> -		sed -ri       's,(RESOURCE_TAR=)n,\1y,' $1/.config
> +		sed -ri=""       's,(RESOURCE_TAR=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_ISAL" != y || \
> -		sed -ri           's,(PMD_ISAL=)n,\1y,' $1/.config
> +		sed -ri=""           's,(PMD_ISAL=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_MLX" != y || \
> -		sed -ri           's,(MLX._PMD=)n,\1y,' $1/.config
> +		sed -ri=""           's,(MLX._PMD=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_SZE" != y || \
> -		sed -ri       's,(PMD_SZEDATA2=)n,\1y,' $1/.config
> +		sed -ri=""       's,(PMD_SZEDATA2=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_ZLIB" != y || \
> -		sed -ri          's,(BNX2X_PMD=)n,\1y,' $1/.config
> +		sed -ri=""          's,(BNX2X_PMD=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_ZLIB" != y || \
> -		sed -ri           's,(PMD_ZLIB=)n,\1y,' $1/.config
> +		sed -ri=""           's,(PMD_ZLIB=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_ZLIB" != y || \
> -		sed -ri   's,(COMPRESSDEV_TEST=)n,\1y,' $1/.config
> +		sed -ri=""   's,(COMPRESSDEV_TEST=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_PCAP" != y || \
> -		sed -ri               's,(PCAP=)n,\1y,' $1/.config
> +		sed -ri=""               's,(PCAP=)n,\1y,' $1/.config
>   		test -z "$ARMV8_CRYPTO_LIB_PATH" || \
> -		sed -ri   's,(PMD_ARMV8_CRYPTO=)n,\1y,' $1/.config
> +		sed -ri=""   's,(PMD_ARMV8_CRYPTO=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_IPSEC_MB" != y || \
> -		sed -ri       's,(PMD_AESNI_MB=)n,\1y,' $1/.config
> +		sed -ri=""       's,(PMD_AESNI_MB=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_IPSEC_MB" != y || \
> -		sed -ri      's,(PMD_AESNI_GCM=)n,\1y,' $1/.config
> +		sed -ri=""      's,(PMD_AESNI_GCM=)n,\1y,' $1/.config
>   		test -z "$LIBSSO_SNOW3G_PATH" || \
> -		sed -ri         's,(PMD_SNOW3G=)n,\1y,' $1/.config
> +		sed -ri=""         's,(PMD_SNOW3G=)n,\1y,' $1/.config
>   		test -z "$LIBSSO_KASUMI_PATH" || \
> -		sed -ri         's,(PMD_KASUMI=)n,\1y,' $1/.config
> +		sed -ri=""         's,(PMD_KASUMI=)n,\1y,' $1/.config
>   		test -z "$LIBSSO_ZUC_PATH" || \
> -		sed -ri            's,(PMD_ZUC=)n,\1y,' $1/.config
> +		sed -ri=""            's,(PMD_ZUC=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_SSL" != y || \
> -		sed -ri            's,(PMD_CCP=)n,\1y,' $1/.config
> +		sed -ri=""            's,(PMD_CCP=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_SSL" != y || \
> -		sed -ri        's,(PMD_OPENSSL=)n,\1y,' $1/.config
> +		sed -ri=""        's,(PMD_OPENSSL=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_SSL" != y || \
> -		sed -ri            's,(QAT_SYM=)n,\1y,' $1/.config
> +		sed -ri=""            's,(QAT_SYM=)n,\1y,' $1/.config
>   		test -z "$FLEXRAN_SDK" || \
> -		sed -ri     's,(BBDEV_TURBO_SW=)n,\1y,' $1/.config
> -		sed -ri           's,(SCHED_.*=)n,\1y,' $1/.config
> +		sed -ri=""     's,(BBDEV_TURBO_SW=)n,\1y,' $1/.config
> +		sed -ri=""           's,(SCHED_.*=)n,\1y,' $1/.config
>   		test -z "$LIBMUSDK_PATH" || \
> -		sed -ri   's,(PMD_MVSAM_CRYPTO=)n,\1y,' $1/.config
> +		sed -ri=""   's,(PMD_MVSAM_CRYPTO=)n,\1y,' $1/.config
>   		test -z "$LIBMUSDK_PATH" || \
> -		sed -ri          's,(MVPP2_PMD=)n,\1y,' $1/.config
> +		sed -ri=""          's,(MVPP2_PMD=)n,\1y,' $1/.config
>   		test -z "$LIBMUSDK_PATH" || \
> -		sed -ri         's,(MVNETA_PMD=)n,\1y,' $1/.config
> +		sed -ri=""         's,(MVNETA_PMD=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_ELF" != y || \
> -		sed -ri            's,(BPF_ELF=)n,\1y,' $1/.config
> +		sed -ri=""            's,(BPF_ELF=)n,\1y,' $1/.config
>   		test "$DPDK_DEP_JSON" != y || \
> -		sed -ri          's,(TELEMETRY=)n,\1y,' $1/.config
> +		sed -ri=""          's,(TELEMETRY=)n,\1y,' $1/.config
>   		build_config_hook $1 $2 $3
>   
>   		# Explicit enabler/disabler (uppercase)
>   		for option in $(echo $3 | sed 's,[~+], &,g') ; do
>   			pattern=$(echo $option | cut -c2-)
>   			if echo $option | grep -q '^~' ; then
> -				sed -ri "s,($pattern=)y,\1n," $1/.config
> +				sed -ri="" "s,($pattern=)y,\1n," $1/.config
>   			elif echo $option | grep -q '^+' ; then
> -				sed -ri "s,($pattern=)n,\1y," $1/.config
> +				sed -ri="" "s,($pattern=)n,\1y," $1/.config
>   			fi
>   		done
>   	fi
> @@ -220,22 +224,22 @@ for conf in $configs ; do
>   	# reload config with DPDK_TARGET set
>   	DPDK_TARGET=$target
>   	reset_env
> -	. $(dirname $(readlink -e $0))/load-devel-config
> +	. $(dirname $(readlink -f $0))/load-devel-config
>   
>   	options=$(echo $conf | sed 's,[^~+]*,,')
>   	dir=$conf
>   	config $dir $target $options
>   
>   	echo "================== Build $dir"
> -	make -j$J EXTRA_CFLAGS="$maxerr $DPDK_DEP_CFLAGS" \
> +	${MAKE} -j$J EXTRA_CFLAGS="$maxerr $DPDK_DEP_CFLAGS" \
>   		EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose O=$dir
>   	! $short || break
>   	echo "================== Build examples for $dir"
>   	export RTE_SDK=$(pwd)
>   	export RTE_TARGET=$dir
> -	make -j$J -sC examples \
> +	${MAKE} -j$J -sC examples \
>   		EXTRA_LDFLAGS="$DPDK_DEP_LDFLAGS" $verbose \
> -		O=$(readlink -m $dir/examples)
> +		O=$(readlink -f $dir/examples)
>   	unset RTE_TARGET
>   	echo "################## $dir done."
>   	unset dir
> @@ -244,9 +248,9 @@ done
>   if ! $short ; then
>   	mkdir -p .check
>   	echo "================== Build doxygen HTML API"
> -	make doc-api-html >/dev/null 2>.check/doc.txt
> +	${MAKE} doc-api-html >/dev/null 2>.check/doc.txt
>   	echo "================== Build sphinx HTML guides"
> -	make doc-guides-html >/dev/null 2>>.check/doc.txt
> +	${MAKE} doc-guides-html >/dev/null 2>>.check/doc.txt
>   	echo "================== Check docs"
>   	diff -u /dev/null .check/doc.txt
>   fi
> diff --git a/devtools/validate-abi.sh b/devtools/validate-abi.sh
> index 138436d93..f64e19d38 100755
> --- a/devtools/validate-abi.sh
> +++ b/devtools/validate-abi.sh
> @@ -181,7 +181,7 @@ case "${dst}" in
>   	/*) ;;
>   	*) dst=${PWD}/${dst} ;;
>   esac
> -dpdkroot=$(readlink -e $(dirname $0)/..)
> +dpdkroot=$(readlink -f $(dirname $0)/..)
>   
>   if [ -e "${dst}" -a "$force" = 0 ]; then
>   	echo "The ${dst} directory is not empty. Remove it, use another


The example applications are being built in a wrong location when using 
the test-build.sh script on FreeBSD 
(“/app_name/x86_64-native-freebsd-gcc/” instead of 
“$(dpdk_path)/x86_64-native-freebsd-gcc/examples/app_name/x86_64-native-freebsd-gcc/”. 


The output build directory for the example applications is specified by 
O=$(readlink -f $dir/examples) variable inside the test-build.sh script. 
On FreeBSD "readlink -f" prints nothing when the file path doesn't 
exist, so the output build directory for each application ends up being 
configured to "/./app_name/x86_64-native-freebsd-gcc" (this is done 
inside the mk/rte.extsubdir.mk file).

A possible solution for this could be to modify the output directory to 
O=$(readlink -f $dir)/examples.

Best Regards,

Flavia
  

On Mon, Jul 08, 2019 at 06:35:25PM +0200, Thomas Monjalon wrote:
> 05/07/2019 15:58, Olivier Matz:
> > - As "readlink -e" and "readlink -m" do not exist on freebsd,
> >   use "readlink -f", it should not have any impact in these cases.
> > - "sed -ri" is invalid on freebsd and should be replaced by
> >   "sed -ri=''"
> > - Use gmake instead of make.
> > 
> > This fixes the following command:
> >   SYSDIR=/usr/src/sys ./devtools/test-build.sh \
> >     -j4 x86_64-native-freebsd-gcc
> > 
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
> > ---
> > --- a/devtools/test-build.sh
> > +++ b/devtools/test-build.sh
> > +[ -z $MAKE ] && command -v gmake > /dev/null && MAKE=gmake
> > +[ -z $MAKE ] && command -v make > /dev/null && MAKE=make
> > +[ -z $MAKE ] && echo "Cannot find make or gmake" && exit 1
> 
> I'm surprised it works, given the -e in this script.
> I would be more confortable with "if/elif" constructs.

Honnestly, I'm surprised too :)

This is the expected. Quoting "man sh":

  -e errexit
       If not interactive, exit immediately if any untested command fails.  The
       exit status of a command is considered to be explicitly tested if the
       command is used to control an if, elif, while, or until; or if the
       command is the left hand operand of an "&&" or "||" operator.
  

Hi,

On Tue, Jul 09, 2019 at 12:05:59PM +0100, Musatescu, Flavia wrote:
> 
> On 05/07/2019 14:58, Olivier Matz wrote:
> > - As "readlink -e" and "readlink -m" do not exist on freebsd,
> >    use "readlink -f", it should not have any impact in these cases.
> > - "sed -ri" is invalid on freebsd and should be replaced by
> >    "sed -ri=''"
> > - Use gmake instead of make.
> > 
> > This fixes the following command:
> >    SYSDIR=/usr/src/sys ./devtools/test-build.sh \
> >      -j4 x86_64-native-freebsd-gcc
> > 
> > Signed-off-by: Olivier Matz <olivier.matz@6wind.com>

(...)

> The example applications are being built in a wrong location when using the
> test-build.sh script on FreeBSD (“/app_name/x86_64-native-freebsd-gcc/”
> instead of “$(dpdk_path)/x86_64-native-freebsd-gcc/examples/app_name/x86_64-native-freebsd-gcc/”.
> 
> 
> The output build directory for the example applications is specified by
> O=$(readlink -f $dir/examples) variable inside the test-build.sh script. On
> FreeBSD "readlink -f" prints nothing when the file path doesn't exist, so
> the output build directory for each application ends up being configured to
> "/./app_name/x86_64-native-freebsd-gcc" (this is done inside the
> mk/rte.extsubdir.mk file).
> 
> A possible solution for this could be to modify the output directory to
> O=$(readlink -f $dir)/examples.

Thanks for spotting this. I will send a v3.