[dpdk-dev,v5] devtools: add tags and cscope index file generation support

  This script generates cscope, gtags, and tags index files based on
EAL environment(architecture and OS(linux/bsd)).

Selection of the architecture and OS environment is based on dpdk
configuration target(T=).If EAL environment(T=) is not specified,
the script generates tag files based on available source code.

Usage: make tags|cscope|gtags|etags [T=config]

example usage:
make cscope
make tags T=x86_64-native-linuxapp-gcc
make gtags T=arm64-armv8a-linuxapp-gcc

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
---
v5:
- Added cscope,tags,gtags,etags makefile targets (Cristian)

v4:
- Accommodate the latest "test" directory rework change in master

v3:
- Added etags target for Emacs(John)
- EAL environment(config) is optional now(Thomas)
- Changed bash shebang to /bin/sh (Thomas)
- getopts based -v and -h option (Thomas)
- used $() instead of backquotes (Thomas)
- Removed "make" based frontend to the script to make it inline with
other devtools scripts in DPDK (Jerin)

v2:
- Moved tag.sh to devtools from scripts
- Rebased to master
---
 .gitignore             |   9 ++
 devtools/build-tags.sh | 271 +++++++++++++++++++++++++++++++++++++++++++++++++
 mk/rte.sdkroot.mk      |   8 ++
 3 files changed, 288 insertions(+)
 create mode 100755 devtools/build-tags.sh
  

-----Original Message-----
> Date: Wed, 22 Mar 2017 14:00:04 +0530
> From: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> To: dev@dpdk.org
> Cc: thomas.monjalon@6wind.com, Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Subject: [dpdk-dev] [PATCH v5] devtools: add tags and cscope index file
>  generation support
> X-Mailer: git-send-email 2.5.5
> 
> This script generates cscope, gtags, and tags index files based on
> EAL environment(architecture and OS(linux/bsd)).
> 
> Selection of the architecture and OS environment is based on dpdk
> configuration target(T=).If EAL environment(T=) is not specified,
> the script generates tag files based on available source code.
> 
> Usage: make tags|cscope|gtags|etags [T=config]
> 
> example usage:
> make cscope
> make tags T=x86_64-native-linuxapp-gcc
> make gtags T=arm64-armv8a-linuxapp-gcc
> 
> Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
> Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>

Ping.


> ---
> v5:
> - Added cscope,tags,gtags,etags makefile targets (Cristian)
> 
> v4:
> - Accommodate the latest "test" directory rework change in master
> 
> v3:
> - Added etags target for Emacs(John)
> - EAL environment(config) is optional now(Thomas)
> - Changed bash shebang to /bin/sh (Thomas)
> - getopts based -v and -h option (Thomas)
> - used $() instead of backquotes (Thomas)
> - Removed "make" based frontend to the script to make it inline with
> other devtools scripts in DPDK (Jerin)
> 
> v2:
> - Moved tag.sh to devtools from scripts
> - Rebased to master
> ---
>  .gitignore             |   9 ++
>  devtools/build-tags.sh | 271 +++++++++++++++++++++++++++++++++++++++++++++++++
>  mk/rte.sdkroot.mk      |   8 ++
>  3 files changed, 288 insertions(+)
>  create mode 100755 devtools/build-tags.sh
> 
> diff --git a/.gitignore b/.gitignore
> index a722abe..7eade38 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -1 +1,10 @@
>  doc/guides/nics/overview_table.txt
> +cscope.out.po
> +cscope.out.in
> +cscope.out
> +cscope.files
> +GTAGS
> +GPATH
> +GRTAGS
> +tags
> +TAGS
> diff --git a/devtools/build-tags.sh b/devtools/build-tags.sh
> new file mode 100755
> index 0000000..ca5317f
> --- /dev/null
> +++ b/devtools/build-tags.sh
> @@ -0,0 +1,271 @@
> +#!/bin/sh -e
> +# Generate tags or gtags or cscope or etags files
> +#
> +#   BSD LICENSE
> +#
> +#   Copyright 2017 Cavium Networks
> +#
> +#   Redistribution and use in source and binary forms, with or without
> +#   modification, are permitted provided that the following conditions
> +#   are met:
> +#
> +#     * Redistributions of source code must retain the above copyright
> +#       notice, this list of conditions and the following disclaimer.
> +#     * Redistributions in binary form must reproduce the above copyright
> +#       notice, this list of conditions and the following disclaimer in
> +#       the documentation and/or other materials provided with the
> +#       distribution.
> +#     * Neither the name of Cavium networks nor the names of its
> +#       contributors may be used to endorse or promote products derived
> +#       from this software without specific prior written permission.
> +#
> +#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> +#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> +#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
> +#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
> +#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
> +#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> +#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> +#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> +#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> +
> +verbose=false
> +linuxapp=false
> +bsdapp=false
> +x86_64=false
> +arm=false
> +arm64=false
> +ia_32=false
> +ppc_64=false
> +
> +print_usage()
> +{
> +	echo "Usage: $(basename $0) [-h] [-v] tags|cscope|gtags|etags [config]"
> +	echo "Valid configs are:"
> +	make showconfigs | sed 's,^,\t,'
> +}
> +
> +while getopts hv ARG ; do
> +	case $ARG in
> +		v ) verbose=true ;;
> +		h ) print_usage; exit 0 ;;
> +		? ) print_usage; exit 1 ;;
> +	esac
> +done
> +shift $(($OPTIND - 1))
> +
> +#ignore version control files
> +ignore="( -name .svn -o -name CVS -o -name .hg -o -name .git ) -prune -o"
> +
> +source_dirs="test app buildtools drivers examples lib"
> +
> +skip_bsd="( -name bsdapp ) -prune -o"
> +skip_linux="( -name linuxapp ) -prune -o"
> +skip_arch="( -name arch ) -prune -o"
> +skip_sse="( -name *_sse*.[chS] ) -prune -o"
> +skip_avx="( -name *_avx*.[chS] ) -prune -o"
> +skip_neon="( -name *_neon*.[chS] ) -prune -o"
> +skip_altivec="( -name *_altivec*.[chS] ) -prune -o"
> +skip_arm64="( -name *arm64*.[chS] ) -prune -o"
> +skip_x86="( -name *x86*.[chS] ) -prune -o"
> +skip_32b_files="( -name *_32.h ) -prune -o"
> +skip_64b_files="( -name *_64.h ) -prune -o"
> +
> +skiplist="$skip_bsd $skip_linux $skip_arch $skip_sse $skip_avx \
> +		 $skip_neon $skip_altivec $skip_x86 $skip_arm64"
> +
> +find_sources()
> +{
> +	find $1 $ignore $3 -name $2 -not -type l -print
> +}
> +
> +common_sources()
> +{
> +	find_sources "$source_dirs" '*.[chS]' "$skiplist"
> +}
> +
> +linuxapp_sources()
> +{
> +	find_sources "lib/librte_eal/linuxapp" '*.[chS]'
> +}
> +
> +bsdapp_sources()
> +{
> +	find_sources "lib/librte_eal/bsdapp" '*.[chS]'
> +}
> +
> +arm_common()
> +{
> +	find_sources "lib/librte_eal/common/arch/arm" '*.[chS]'
> +	find_sources "$source_dirs" '*neon*.[chS]'
> +}
> +
> +arm_sources()
> +{
> +	arm_common
> +	find_sources "lib/librte_eal/common/include/arch/arm" '*.[chS]' \
> +					"$skip_64b_files"
> +}
> +
> +arm64_sources()
> +{
> +	arm_common
> +	find_sources "lib/librte_eal/common/include/arch/arm" '*.[chS]' \
> +					 "$skip_32b_files"
> +	find_sources "$source_dirs" '*arm64.[chS]'
> +}
> +
> +ia_common()
> +{
> +	find_sources "lib/librte_eal/common/arch/x86" '*.[chS]'
> +
> +	find_sources "examples/performance-thread/common/arch/x86" '*.[chS]'
> +	find_sources "$source_dirs" '*_sse*.[chS]'
> +	find_sources "$source_dirs" '*_avx*.[chS]'
> +	find_sources "$source_dirs" '*x86.[chS]'
> +}
> +
> +i686_sources()
> +{
> +	ia_common
> +	find_sources "lib/librte_eal/common/include/arch/x86" '*.[chS]' \
> +					"$skip_64b_files"
> +}
> +
> +x86_64_sources()
> +{
> +	ia_common
> +	find_sources "lib/librte_eal/common/include/arch/x86" '*.[chS]' \
> +					"$skip_32b_files"
> +}
> +
> +ppc64_sources()
> +{
> +	find_sources "lib/librte_eal/common/arch/ppc_64" '*.[chS]'
> +	find_sources "lib/librte_eal/common/include/arch/ppc_64" '*.[chS]'
> +	find_sources "$source_dirs" '*altivec*.[chS]'
> +}
> +
> +config_file()
> +{
> +	if [ -f $RTE_SDK/$RTE_TARGET/include/rte_config.h ]; then
> +		ls $RTE_SDK/$RTE_TARGET/include/rte_config.h
> +	fi
> +}
> +
> +check_valid_config()
> +{
> +	cfgfound=false
> +	allconfigs=$(make showconfigs)
> +	for cfg in $allconfigs
> +	do
> +		if [ "$cfg" = "$1" ] ; then
> +			cfgfound=true
> +		fi
> +	done
> +	$cfgfound || echo "Invalid config: $1"
> +	$cfgfound || print_usage
> +	$cfgfound || exit 0
> +}
> +
> +if [ -n "$2" ]; then
> +	check_valid_config $2
> +
> +	if [ $(echo $2 | grep -c "linuxapp-") -gt 0 ]; then
> +		linuxapp=true
> +	fi
> +
> +	if [ $(echo $2 | grep -c "bsdapp-") -gt 0 ]; then
> +		bsdapp=true
> +	fi
> +
> +	if [ $(echo $2 | grep -c "x86_64-") -gt 0  ]; then
> +		x86_64=true
> +	fi
> +
> +	if [ $(echo $2 | grep -c "i686-") -gt 0 ]; then
> +		ia_32=true
> +	fi
> +
> +	if [ $(echo $2 | grep -c "x32-") -gt 0 ]; then
> +		ia_32=true
> +	fi
> +
> +	if [ $(echo $2 | grep -c "arm-") -gt 0 ]; then
> +		arm=true
> +	fi
> +
> +	if [ $(echo $2 | grep -c "arm64-") -gt 0 ]; then
> +		arm64=true
> +	fi
> +
> +	if [ $(echo $2 | grep -c "ppc_64-") -gt 0 ]; then
> +		ppc_64=true
> +	fi
> +
> +else
> +	linuxapp=true
> +	bsdapp=true
> +	x86_64=true
> +	arm=true
> +	arm64=true
> +	ia_32=true
> +	ppc_64=true
> +fi
> +
> +all_sources()
> +{
> +	common_sources
> +	$linuxapp && linuxapp_sources
> +	$bsdapp && bsdapp_sources
> +	$x86_64 && x86_64_sources
> +	$ia_32 && i686_sources
> +	$arm && arm_sources
> +	$arm64 && arm64_sources
> +	$ppc_64 && ppc64_sources
> +	config_file
> +}
> +
> +show_flags()
> +{
> +	$verbose && echo "mode:     $1"
> +	$verbose && echo "config:   $2"
> +	$verbose && echo "linuxapp: $linuxapp"
> +	$verbose && echo "bsdapp:   $bsdapp"
> +	$verbose && echo "ia_32:    $ia_32"
> +	$verbose && echo "x86_64:   $x86_64"
> +	$verbose && echo "arm:      $arm"
> +	$verbose && echo "arm64:    $arm64"
> +	$verbose && echo "ppc_64:   $ppc_64"
> +	if [ -f $RTE_SDK/$RTE_TARGET/include/rte_config.h ]; then
> +		$verbose && echo "target:   $RTE_SDK/$RTE_TARGET"
> +	fi
> +}
> +
> +case "$1" in
> +	"cscope")
> +		show_flags $1 $2
> +		all_sources > cscope.files
> +		cscope -q -b -f cscope.out
> +		;;
> +	"gtags")
> +		show_flags $1 $2
> +		all_sources | gtags -i -f -
> +		;;
> +	"tags")
> +		show_flags $1 $2
> +		rm -f tags
> +		all_sources | xargs ctags -a
> +		;;
> +	"etags")
> +		show_flags $1 $2
> +		rm -f TAGS
> +		all_sources | xargs etags -a
> +		;;
> +	*)
> +		echo "Invalid mode: $1"
> +		print_usage
> +		;;
> +esac
> diff --git a/mk/rte.sdkroot.mk b/mk/rte.sdkroot.mk
> index 7598bde..6f9bfc5 100644
> --- a/mk/rte.sdkroot.mk
> +++ b/mk/rte.sdkroot.mk
> @@ -92,6 +92,14 @@ default: all
>  config showconfigs showversion showversionum:
>  	$(Q)$(MAKE) -f $(RTE_SDK)/mk/rte.sdkconfig.mk $@
>  
> +.PHONY: cscope gtags tags etags
> +cscope gtags tags etags:
> +ifdef T
> +	$(Q)$(RTE_SDK)/devtools/build-tags.sh $@ ${T}
> +else
> +	$(Q)$(RTE_SDK)/devtools/build-tags.sh $@
> +endif
> +
>  .PHONY: test test-basic test-fast test-ring test-mempool test-perf coverage
>  test test-basic test-fast test-ring test-mempool test-perf coverage:
>  	$(Q)$(MAKE) -f $(RTE_SDK)/mk/rte.sdktest.mk $@
> -- 
> 2.5.5
>
  

Hi,
I have some comments about behaviour, naming and coding style.

22/03/2017 09:30, Jerin Jacob:
> +verbose=false
> +linuxapp=false
> +bsdapp=false

linuxapp/bsdapp could be renamed to simpler linux/bsd.

> +x86_64=false
> +arm=false

Why not arm32 for consistency?

> +arm64=false
> +ia_32=false

ia_32 should be x86_32

> +ppc_64=false

[...]
> +	make showconfigs | sed 's,^,\t,'

This is assuming the working dir is the dpdk root.
It is assumed elsewhere in the script.
I suggest adding "cd $(dirname $0)/.." at the beginning of the script.

[...]
> +arm_common()
> +{
> +	find_sources "lib/librte_eal/common/arch/arm" '*.[chS]'
> +	find_sources "$source_dirs" '*neon*.[chS]'
> +}
> +
> +arm_sources()
> +{
> +	arm_common
> +	find_sources "lib/librte_eal/common/include/arch/arm" '*.[chS]' \
> +					"$skip_64b_files"
> +}
> +
> +arm64_sources()
> +{
> +	arm_common
> +	find_sources "lib/librte_eal/common/include/arch/arm" '*.[chS]' \
> +					 "$skip_32b_files"
> +	find_sources "$source_dirs" '*arm64.[chS]'
> +}

Same comment about arm32 consistency.

> +ia_common()
> +{
> +	find_sources "lib/librte_eal/common/arch/x86" '*.[chS]'
> +
> +	find_sources "examples/performance-thread/common/arch/x86" '*.[chS]'
> +	find_sources "$source_dirs" '*_sse*.[chS]'
> +	find_sources "$source_dirs" '*_avx*.[chS]'
> +	find_sources "$source_dirs" '*x86.[chS]'
> +}
> +
> +i686_sources()
> +{
> +	ia_common
> +	find_sources "lib/librte_eal/common/include/arch/x86" '*.[chS]' \
> +					"$skip_64b_files"
> +}
> +
> +x86_64_sources()
> +{
> +	ia_common
> +	find_sources "lib/librte_eal/common/include/arch/x86" '*.[chS]' \
> +					"$skip_32b_files"
> +}

... and here x86/x86_32 instead of ia/i686


> +config_file()
> +{
> +	if [ -f $RTE_SDK/$RTE_TARGET/include/rte_config.h ]; then
> +		ls $RTE_SDK/$RTE_TARGET/include/rte_config.h
> +	fi
> +}

RTE_TARGET is used to define the build directory.
It will not be defined in this script and it does not really
make sense to enforce it.
I suggest to remove rte_config.h from this script.
Other argument: you can have several build directories
with different configs.

[...]
> +check_valid_config()

It is not a config but a target.
Yes the target is checked against existing config template names.

> +{
> +	cfgfound=false
> +	allconfigs=$(make showconfigs)
> +	for cfg in $allconfigs
> +	do

This "do" keyword could be on the previous line, like you did for if...then.

> +		if [ "$cfg" = "$1" ] ; then
> +			cfgfound=true
> +		fi
> +	done
> +	$cfgfound || echo "Invalid config: $1"
> +	$cfgfound || print_usage
> +	$cfgfound || exit 0

Better to use an "if" block here.

[...]
> +	if [ $(echo $2 | grep -c "linuxapp-") -gt 0 ]; then
> +		linuxapp=true
> +	fi

More compact grep:
	if echo $2 | grep -q "linuxapp-" ; then

[...]
> +else
> +	linuxapp=true
> +	bsdapp=true
> +	x86_64=true
> +	arm=true
> +	arm64=true
> +	ia_32=true
> +	ppc_64=true
> +fi

You could define them as true by default.
So the above grep would become:
	echo $2 | grep -q "linuxapp-" || linux=false

[...]
> +all_sources()
> +{
> +	common_sources
> +	$linuxapp && linuxapp_sources
> +	$bsdapp && bsdapp_sources
> +	$x86_64 && x86_64_sources
> +	$ia_32 && i686_sources
> +	$arm && arm_sources
> +	$arm64 && arm64_sources
> +	$ppc_64 && ppc64_sources

I am a bit surprised by these constructs.
When using sh -e, we should not have a false statement.
I think it should be:
	if $linuxapp ; then linuxapp_sources ; fi

[...]
> +show_flags()
> +{
> +	$verbose && echo "mode:     $1"
> +	$verbose && echo "config:   $2"
> +	$verbose && echo "linuxapp: $linuxapp"
> +	$verbose && echo "bsdapp:   $bsdapp"
> +	$verbose && echo "ia_32:    $ia_32"
> +	$verbose && echo "x86_64:   $x86_64"
> +	$verbose && echo "arm:      $arm"
> +	$verbose && echo "arm64:    $arm64"
> +	$verbose && echo "ppc_64:   $ppc_64"

Please use an "if" block here.

> +++ b/mk/rte.sdkroot.mk
> +.PHONY: cscope gtags tags etags
> +cscope gtags tags etags:
> +ifdef T
> +	$(Q)$(RTE_SDK)/devtools/build-tags.sh $@ ${T}
> +else
> +	$(Q)$(RTE_SDK)/devtools/build-tags.sh $@
> +endif

No need of ifdef.

Thanks Jerin