Re: Snort 3: Add --disable-txt-docs option to configure_cmake.sh patches

Michael Altizer via Snort-devel <[email protected]> Tue, 20 Mar 2018 10:45:45 -0400

Newsgroups	gmane.comp.security.ids.snort.devel
Message-ID	<[email protected]>

Ok, thanks.Â  I've attached a patch that should remove the bashisms from 
all but one of the scripts (which won't execute normally) if you would 
like to try it.Â  We'll also pull in your patch since it seems like a 
useful option to have even if the scripts are working.

On 03/19/2018 11:22 AM, Y M via Snort-devel wrote:
>
> Hi Michael,
>
>
> Tested with #!/bin/sh on CentOS and FreeBSD. On CentOS it went okay. 
> On FreeBSD, the make output is shown below. Doing an install 
> afterwards did not create the $prefix/doc directory.
>
>
> [98%] Built target snort
> Scanning dependencies of target all_built_sources
> [ 98%] Documents: building config.txt with
> [ 98%] Documents: building counts.txt with
> [ 98%] Documents: building modules.txt with
> [ 98%] Documents: building plugins.txt with
> [ 98%] Documents: building signals.txt with
> ../../doc/scripts/generate_help.sh: [[: not found
> [ 99%] Documents: building builtin.txt with
> ../../doc/scripts/generate_help.sh: [[: not found
> [ 99%] Documents: building help.txt with
> [ 99%] Documents: building options.txt with
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_list.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> [ 99%] Documents: building gids.txt with
> [ 99%] Documents: building codec.txt with
> [ 99%] Documents: building connector.txt with
> ../../doc/scripts/generate_list.sh: [[: not found
> [ 99%] Documents: building basic.txt with
> ../../doc/scripts/generate_module.sh: [[: not found
> ../../doc/scripts/generate_module.sh: [[: not found
> [ 99%] Documents: building data.txt with
> [ 99%] Documents: building ips_action.txt with
> [100%] Documents: building inspector.txt with
> [100%] Documents: building ips_option.txt with
> ../../doc/scripts/generate_module.sh: [[: not found
> ../../doc/scripts/generate_module.sh: [[: not found
> ../../doc/scripts/generate_module.sh: [[: not found
> ../../doc/scripts/generate_module.sh: [[: not found
> ../../doc/scripts/generate_module.sh: [[: not found
> [100%] Documents: building logger.txt with
> ../../doc/scripts/generate_module.sh: [[: not found
> [100%] Documents: building version.txt
> [100%] Documents: building commands.txt with
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> ../../doc/scripts/generate_help.sh: [[: not found
> [100%] Documents: building snort2lua_cmds.txt
> [100%] Built target all_built_sources
> Scanning dependencies of target snort_manuals
> [100%] Built target snort_manuals
>
> ------------------------------------------------------------------------
> *From:* Snort-devel <[email protected]> on behalf of 
> Michael Altizer via Snort-devel <[email protected]>
> *Sent:* Monday, March 19, 2018 5:49:50 PM
> *To:* [email protected]
> *Subject:* Re: [Snort-devel] Snort 3: Add --disable-txt-docs option to 
> configure_cmake.sh patches
> On 03/16/2018 10:32 AM, Y M via Snort-devel wrote:
>>
>> Hi,
>>
>>
>> The attached patch files will add a new option --disable-txt-docs to 
>> the configure_cmake.sh script, and wrap the generation of Snort 
>> $prefix/doc/* in doc/CMakeList.txt with logic to check if the new 
>> option is set. If the option is used, the text files and the 
>> $prefix/doc directory will not be generated. This is possible thanks 
>> to the existing logic to handle MAKE_TEXT_DOC.
>>
>>
>> This change stems from the fact that the scripts: 
>> doc/scripts/generate_help.sh, doc/scripts/generate_list.sh, and 
>> doc/scripts/generate_module.sh use bash "#!/usr/bin/env bash" for the 
>> shell. On a pristine FreeBSD image or in cases bash is not installed, 
>> the make script will fail with the below error with no obvious way on 
>> how to proceed with compiling Snort 3. This occurs in Snort builds 
>> 243 and 244.
>>
>>
>> Scanningdependencies of target codecs
>> [ 97%] Building CXX object 
>> src/codecs/CMakeFiles/codecs.dir/codec_api.cc.o
>> [ 98%] Linking CXX static library libcodecs.a
>> [ 98%] Built target codecs
>> Scanning dependencies of target snort
>> [ 98%] Building CXX object src/CMakeFiles/snort.dir/main.cc.o
>> [ 98%] Linking CXX executable snort
>> [ 98%] Built target snort
>> Scanning dependencies of target all_built_sources
>> [ 98%] Documents: building counts.txt with
>> [ 98%] Documents: building modules.txt with
>> [ 98%] Documents: building plugins.txt with
>> [ 98%] Documents: building signals.txt with
>> [ 98%] Documents: building builtin.txt with
>> [ 98%] Documents: building options.txt with
>> [ 98%] Documents: building config.txt with
>> env: bash: No such file or directory
>> --- doc/counts.txt ---
>> *** [doc/counts.txt] Error code 127
>>
>> ...
>>
>>
>> With the changes applied, Snort builds successfully. I am not sure if 
>> this is the best way to handle it, but it avoids missing with the 
>> shell scripts portability among the different shells.
>>
>>
>> Thanks.
>>
>> YM
>>
>>
>
> Thanks for the patch, but since you seem to have a bash-less FreeBSD 
> system handy, could you try just changing them all to use #!/bin/sh 
> and see if there were any actual bash-isms?Â  If not, we'll just swap 
> the interpreter.
>
>
> _______________________________________________
> Snort-devel mailing list
> [email protected]
> https://lists.snort.org/mailman/listinfo/snort-devel
>
> Please visit http://blog.snort.org for the latest news about Snort!

_______________________________________________
Snort-devel mailing list
[email protected]
https://lists.snort.org/mailman/listinfo/snort-devel

Please visit http://blog.snort.org for the latest news about Snort!

build-Remove-bashisms-from-most-shell-scripts.patch (text/x-patch, 5.3 KB)

From 6e04230b31d4a8ab6c79c2e8db24c02a6dbb72c8 Mon Sep 17 00:00:00 2001
From: Michael Altizer <[email protected]>
Date: Mon, 19 Mar 2018 21:18:12 -0400
Subject: [PATCH] build: Remove bashisms from most shell scripts

---
 configure_cmake.sh             |  3 +--
 doc/dev_guide.sh               |  2 +-
 doc/online_manual.sh           |  6 +++---
 doc/scripts/generate_help.sh   |  8 ++++----
 doc/scripts/generate_list.sh   |  4 ++--
 doc/scripts/generate_module.sh |  4 ++--
 extra/configure_cmake.sh       |  3 +--
 extra/generate_boilerplate.sh  | 20 ++++++++------------
 8 files changed, 22 insertions(+), 28 deletions(-)

diff --git a/configure_cmake.sh b/configure_cmake.sh
index 959d81e43..52bef51f9 100755
--- a/configure_cmake.sh
+++ b/configure_cmake.sh
@@ -400,8 +400,7 @@ echo "Build Directory : $builddir"
 echo "Source Directory: $sourcedir"
 cd $builddir
 
-gen=""
-[ "$CMakeGenerator" ] && gen+=" -G $CMakeGenerator"
+[ "$CMakeGenerator" ] && gen="-G $CMakeGenerator"
 
 cmake $gen \
     -DCMAKE_CXX_FLAGS:STRING="$CXXFLAGS $CPPFLAGS" \
diff --git a/doc/dev_guide.sh b/doc/dev_guide.sh
index c794c40d2..5ec4477b6 100755
--- a/doc/dev_guide.sh
+++ b/doc/dev_guide.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/usr/bin/env bash
 # run from top of source tree (where configure.ac lives)
 # or supply top_srcdir as argument
 # this will create /tmp/dev_guide.{txt,html}
diff --git a/doc/online_manual.sh b/doc/online_manual.sh
index 393197cfa..e57b1d35f 100755
--- a/doc/online_manual.sh
+++ b/doc/online_manual.sh
@@ -1,4 +1,4 @@
-#!/bin/bash
+#!/bin/sh
 
 # run this from build/doc/ to create an all in one html manual
 # with embedded images in base64 format for use in places where
@@ -38,11 +38,11 @@ END
 }
 
 # create working dir
-mkdir tmp || exit -1
+mkdir tmp || exit 1
 cd tmp
 
 # copy the sources since we need to make edits
-cp ../*.{txt,png} ./
+cp ../*.txt ../*.png ./
 
 # create a list of 'txt png' pairs
 grep -o 'image::.*.png' *.txt > $data
diff --git a/doc/scripts/generate_help.sh b/doc/scripts/generate_help.sh
index e516bee05..b6b704777 100755
--- a/doc/scripts/generate_help.sh
+++ b/doc/scripts/generate_help.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 
 SNORT_BINARY="$1"
 OUTPUT_FILE="$2"
@@ -7,13 +7,13 @@ HELP_ARG="--help"
 PLUGIN_ARGS=
 SORT_ARGS=
 
-if [[ -n "${PLUGIN_PATH}" ]]; then
+if [ -n "${PLUGIN_PATH}" ] ; then
   PLUGIN_ARGS="--plugin-path=${PLUGIN_PATH}"
 fi
 
 HELP_TYPE=`basename "${OUTPUT_FILE}" .txt`
 
-if [[ "${HELP_TYPE}" != "help" ]]; then
+if [ "${HELP_TYPE}" != "help" ] ; then
   HELP_ARG="${HELP_ARG}-${HELP_TYPE}"
 
   case "${HELP_TYPE}" in
@@ -26,7 +26,7 @@ if [[ "${HELP_TYPE}" != "help" ]]; then
   esac
 fi
 
-if [[ -n "${SORT_ARGS}" ]]; then
+if [ -n "${SORT_ARGS}" ] ; then
   "${SNORT_BINARY}" ${PLUGIN_ARGS} --markup "${HELP_ARG}" | sort ${SORT_ARGS} \
     > "${OUTPUT_FILE}"
 else
diff --git a/doc/scripts/generate_list.sh b/doc/scripts/generate_list.sh
index 3a6f125cc..6cbc62fce 100755
--- a/doc/scripts/generate_list.sh
+++ b/doc/scripts/generate_list.sh
@@ -1,4 +1,4 @@
-#!/usr/bin/env bash
+#!/bin/sh
 
 SNORT_BINARY="$1"
 OUTPUT_FILE="$2"
@@ -7,7 +7,7 @@ LIST_ARG=
 PLUGIN_ARGS=
 SORT_OPTIONS=
 
-if [[ -n "${PLUGIN_PATH}" ]]; then
+if [ -n "${PLUGIN_PATH}" ] ; then
   PLUGIN_ARGS="--plugin-path=${PLUGIN_PATH}"
 fi
 
diff --git a/doc/scripts/generate_module.sh b/doc/scripts/generate_module.sh
index 34d23760d..119810421 100755
--- a/doc/scripts/generate_module.sh
+++ b/doc/scripts/generate_module.sh
@@ -1,11 +1,11 @@
-#!/usr/bin/env bash
+#!/bin/sh
 
 SNORT_BINARY="$1"
 OUTPUT_FILE="$2"
 PLUGIN_PATH="$3"
 PLUGIN_ARGS=
 
-if [[ -n "${PLUGIN_PATH}" ]]; then
+if [ -n "${PLUGIN_PATH}" ] ; then
   PLUGIN_ARGS="--plugin-path=${PLUGIN_PATH}"
 fi
 
diff --git a/extra/configure_cmake.sh b/extra/configure_cmake.sh
index 882d2ed39..f201ea62c 100755
--- a/extra/configure_cmake.sh
+++ b/extra/configure_cmake.sh
@@ -86,8 +86,7 @@ echo "Build Directory : $builddir"
 echo "Source Directory: $sourcedir"
 cd $builddir
 
-gen=""
-[ "$CMakeGenerator" ] && gen+=" -G $CMakeGenerator"
+[ "$CMakeGenerator" ] && gen="-G $CMakeGenerator"
 
 cmake $gen \
     -DCMAKE_CXX_FLAGS:STRING="$CXXFLAGS $CPPFLAGS" \
diff --git a/extra/generate_boilerplate.sh b/extra/generate_boilerplate.sh
index 9fef33850..fae591de3 100755
--- a/extra/generate_boilerplate.sh
+++ b/extra/generate_boilerplate.sh
@@ -1,8 +1,8 @@
-#!/bin/bash
+#!/bin/sh
 
 USAGE="Usage: $0 <start> <expander:./scripts> <templates:./templates>"
 
-if [[ $1 == -h ]]; then
+if [ "$1" = "-h" ]; then
   echo $USAGE >&2
   exit
 fi
@@ -13,20 +13,16 @@ TEMPLATE_PATH=./templates
 
 EXPAND=${SCRIPT_PATH}/expand_template.rb
 
-boilerplate=(
-  ${TEMPLATE_PATH}/CMakeLists.txt.erb
-)
+template=${TEMPLATE_PATH}/CMakeLists.txt.erb
 
-[[ -n $DRY_RUN ]] && ECHO=echo || ECHO=
+[ -n "$DRY_RUN" ] && ECHO=echo || ECHO=
 
 for project_dir in $(find $RECURSE -mindepth 3 -type d); do
   project_base=${project_dir##*/}
 
-  for template in "${boilerplate[@]}"; do
-    template_base=${template##*/}
+  template_base=${template##*/}
 
-    [[ -n $DRY_RUN ]] && \
-      echo $EXPAND $template $project_dir '>' $project_dir/${template_base%.erb} || \
-      $EXPAND $template $project_dir > $project_dir/${template_base%.erb}
-  done
+  [ -n "$DRY_RUN" ] && \
+    echo $EXPAND $template $project_dir '>' $project_dir/${template_base%.erb} || \
+    $EXPAND $template $project_dir > $project_dir/${template_base%.erb}
 done
-- 
2.16.2