Re: [PATCH RFC] Add Travis and Appveyor CI support
Keith Mendoza <[email protected]>
| Newsgroups | gmane.network.net-snmp.devel |
|---|---|
| Message-ID | <1524678892.862105.1350621960.0CC2035E@webmail.messagingengine.com> |
Bart,
Out of curiosity, do you have a "fork" of Net-SNMP on github to connect it to Travis and Appveyor?
On Wed, Apr 25, 2018, at 8:06 AM, Bart Van Assche wrote:
> Hello,
>
> One of the advantages of github over SourceForge is that integration
> with continuous integration (CI) services like Travis and Appveyor is
> easy. Adding such support however requires to add proper configuration
> files and the necessary scripts in the source tree. Hence this patch. As
> one can see for Linux all regression tests are run, for OS/X some
> regression tests are run and for MSVC and Cygwin no regression tests are
> run. All four builds pass with this patch. As usual, feedback is welcome.
>
> Bart.
>
>
> ---
> .appveyor.yml | 10 +
> .travis.yml | 35 ++++
> ci/build.bat | 30 +++
> ci/build.sh | 17 ++
> ci/net-snmp-configure | 211 +++++++++++++
> ci/net-snmp-run-perl-tests | 9 +
> ci/net-snmp-run-python-tests | 16 ++
> ci/net-snmp-run-tests | 37 ++++
> .../fulltests/default/T200snmpv2cwalkall_simple | 2 +
> testing/fulltests/support/simple_eval_tools.sh | 21 +-
> 10 files changed, 381 insertions(+), 7 deletions(-)
> create mode 100644 .appveyor.yml
> create mode 100644 .travis.yml
> create mode 100644 ci/build.bat
> create mode 100755 ci/build.sh
> create mode 100755 ci/net-snmp-configure
> create mode 100755 ci/net-snmp-run-perl-tests
> create mode 100755 ci/net-snmp-run-python-tests
> create mode 100755 ci/net-snmp-run-tests
>
> diff --git a/.appveyor.yml b/.appveyor.yml
> new file mode 100644
> index 000000000000..137c52052d34
> --- /dev/null
> +++ b/.appveyor.yml
> @@ -0,0 +1,10 @@
> +image:
> + - Visual Studio 2017
> +environment:
> + matrix:
> + - BUILD: MSVC
> + # - BUILD: MinGW
> + - BUILD: Cygwin
> +clone_depth: 5
> +build_script:
> + - 'call "ci\build.bat"'
> diff --git a/.travis.yml b/.travis.yml
> new file mode 100644
> index 000000000000..7262aa19aab0
> --- /dev/null
> +++ b/.travis.yml
> @@ -0,0 +1,35 @@
> +language: c
> +
> +os:
> + - linux
> + - osx
> +
> +dist: trusty
> +
> +sudo: required
> +
> +addons:
> + apt:
> + packages:
> + - dpkg
> + - gcc
> + - libatm-dev
> + - libperl-dev
> + - libsensors4-dev
> + - libssh-dev
> + - libssl-dev
> + - make
> + - perl-modules
> + - pkg-config
> + - python-dev
> + - python-setuptools
> +
> +# Add an IPv6 config - see the corresponding Travis issue
> +# https://github.com/travis-ci/travis-ci/issues/8361
> +before_script:
> + - if [ "${TRAVIS_OS_NAME}" == "linux" ]; then
> + sudo sh -c 'echo 0 > /proc/sys/net/ipv6/conf/all/disable_ipv6;
> printf "\n::1 localhost ipv6-localhost ipv6-loopback\n" >>/etc/hosts;
> cat /etc/hosts';
> + fi
> + - 'if [ "${TRAVIS_OS_NAME}" == "osx" ]; then brew upgrade openssl
> >/dev/null 2>&1; fi'
> +
> +script: ci/build.sh
> diff --git a/ci/build.bat b/ci/build.bat
> new file mode 100644
> index 000000000000..cea6a89a4886
> --- /dev/null
> +++ b/ci/build.bat
> @@ -0,0 +1,30 @@
> +echo "Build type %BUILD%"
> +@echo on
> +goto %BUILD%
> +echo "Error: unknown build type %BUILD%"
> +goto eof
> +
> +:MSVC
> +REM see also https://www.appveyor.com/docs/lang/cpp/
> +REM if exist "C:\Program Files (x86)\Microsoft Visual Studio\2017" (
> +REM set "d=C:\Program Files (x86)\Microsoft Visual Studio\2017"
> +REM ) else if exist "C:\Program Files\Microsoft Visual Studio\2017" (
> +REM set "d=C:\Program Files\Microsoft Visual Studio\2017"
> +REM )
> +REM cmd /c ""%d%"\Community\VC\Auxiliary\Build\vcvarsall.bat x86"
> +REM install:
> +call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC
> \vcvarsall.bat"
> +cd win32
> +perl Configure --config=release --with-sdk --with-ipv6 --with-winextdll
> --linktype=dynamic
> +nmake
> +goto eof
> +
> +:MinGW
> +C:\msys64\usr\bin\bash --login -c 'set -x; cd
> "${APPVEYOR_BUILD_FOLDER}"; ci/build.sh'
> +goto eof
> +
> +:Cygwin
> +c:\cygwin\bin\bash --login -c 'set -x; cd "${APPVEYOR_BUILD_FOLDER}";
> ci/build.sh'
> +goto eof
> +
> +:eof
> diff --git a/ci/build.sh b/ci/build.sh
> new file mode 100755
> index 000000000000..3e306d82bf17
> --- /dev/null
> +++ b/ci/build.sh
> @@ -0,0 +1,17 @@
> +#!/bin/bash
> +
> +scriptdir="$(dirname "$0")"
> +export NOAUTODEPS=1
> +export SNMP_VERBOSE=1
> +if [ -z "$OSTYPE" ]; then
> + case "$(uname)" in
> + Linux) OSTYPE=linux;;
> + Darwin) OSTYPE=darwin;;
> + *) OSTYPE="UNKNOWN:$(uname)";;
> + esac
> + export OSTYPE
> +fi
> +"${scriptdir}"/net-snmp-configure master || exit $?
> +make -s || exit $?
> +[ "$OSTYPE" = cygwin ] && exit 0
> +"${scriptdir}"/net-snmp-run-tests || exit $?
> diff --git a/ci/net-snmp-configure b/ci/net-snmp-configure
> new file mode 100755
> index 000000000000..03e12e7fe737
> --- /dev/null
> +++ b/ci/net-snmp-configure
> @@ -0,0 +1,211 @@
> +#!/usr/bin/env bash
> +
> +# Necessary software packages for building Net-SNMP:
> +# autoconf automake libtool m4
> +# glibc glibc-devel libssh2 libssh2-devel openssl openssl-devel rpm-
> devel
> +# perl-ExtUtils-MakeMaker perl-ExtUtils-Embed perl-Test-Harness
> +# python python-devel python-setuptools
> +
> +branch_name="$1"
> +shift
> +
> +is_5_7_or_above="false"
> +is_5_8_or_above="false"
> +if [ "${branch_name#master}" != "${branch_name}" ]; then
> + is_5_8_or_above="true"
> + is_5_7_or_above="true"
> +elif [ "${branch_name#V5-7-}" != "${branch_name}" ]; then
> + is_5_7_or_above="true"
> +fi
> +have_rtnetlink=true
> +set -x
> +if ! echo "$*" | grep -Eq 'with-openssl=(no|internal)'; then
> + case "${OSTYPE}" in
> + linux*)
> + opensslver="$(
> + for p in openssl-devel libopenssl-devel libressl-devel;
> do
> + rpm -q --qf '%{version}\n' $p 2>/dev/null;
> + done;
> + dpkg-query --show -f '${Version}\n' libssl-dev 2>/dev/null;
> + pacman -Q openssl 2>&1 | sed -n 's/^openssl[[:blank:]]*//p'
> + )";;
> + *)
> + opensslver=1.0;;
> + esac
> +fi
> +set +x
> +if grep -q 'CentOS release 4' /etc/issue 2>/dev/null; then
> + opensslver=""
> + have_rtnetlink=false
> +fi
> +opensshver="$({
> + rpm -q openssh-devel;
> + rpm -q libssh2-devel;
> + rpm -q libssh-devel; } 2>/dev/null |
> + sed -n 's/^.*ssh[0-9]*-devel-\([0-9a-z.]*\)-.*$/\1/p' | head -n1)"
> +
> +options=()
> +options+=(--enable-developer)
> +options+=(--enable-ipv6)
> +options+=("--prefix=/usr/local/net-snmp-${branch_name}")
> +options+=("--with-cflags=$cflags")
> +options+=(--with-defaults)
> +transports=()
> +[ "$opensshver" != "" ] && transports+=(SSH)
> +# Mib names can be found as follows:
> +# (cd agent/mibgroup &&
> +# git grep -lE 'void[[:blank:]]*init_.*\(void\)|config_require\(' |
> +# sed -n 's/\.h$//p')
> +mibs=()
> +mibs+=(examples/data_set)
> +mibs+=(examples/delayed_instance)
> +mibs+=(examples/example)
> +mibs+=(examples/notification)
> +mibs+=(examples/scalar_int)
> +mibs+=(examples/ucdDemoPublic)
> +mibs+=(examples/watched)
> +mibs+=(mibII)
> +mibs+=(smux)
> +
> +case "$(uname)" in
> + Linux*)
> + options+=(--enable-new-features)
> + $is_5_7_or_above && mibs+=(deliver/deliverByNotify)
> + mibs+=(disman/event)
> + #mibs+=(disman/expression) - to do: figure out why this fails on
> Travis
> + mibs+=(disman/mteEventNotificationTable)
> + mibs+=(disman/mteEventTable)
> + mibs+=(disman/mteObjectsTable)
> + mibs+=(disman/mteTriggerBooleanTable)
> + mibs+=(disman/mteTriggerDeltaTable)
> + mibs+=(disman/mteTriggerExistenceTable)
> + mibs+=(disman/mteTriggerTable)
> + mibs+=(disman/mteTriggerThresholdTable)
> + mibs+=(disman/nslookup-mib)
> + mibs+=(disman/ping-mib)
> + mibs+=(disman/schedule)
> + mibs+=(disman/traceroute-mib)
> + $have_rtnetlink && mibs+=(etherlike-mib)
> + mibs+=(examples/netSnmpHostsTable)
> + mibs+=(hardware/cpu)
> + mibs+=(hardware/fsys)
> + mibs+=(hardware/memory)
> + mibs+=(hardware/sensors)
> + mibs+=(host)
> + mibs+=(ip-forward-mib)
> + mibs+=(ip-mib/inetNetToMediaTable)
> + mibs+=(ip-mib/ipDefaultRouterTable)
> + mibs+=(ip-mib/ipv4InterfaceTable)
> + mibs+=(ip-mib/ipv6InterfaceTable)
> + mibs+=(ip-mib/ipv6ScopeZoneIndexTable)
> + mibs+=(mibII/ipAddr)
> + mibs+=(mibII/mta_sendmail)
> + mibs+=(misc/ipfwacc)
> + mibs+=(sctp-mib)
> + mibs+=(snmp-notification-mib)
> + [ "$opensslver" != "" ] && mibs+=(snmp-usm-dh-objects-mib)
> + mibs+=(tcp-mib)
> + mibs+=(testhandler)
> + [ "$opensslver" != "" ] && mibs+=(tlstm-mib tsm-mib)
> + mibs+=(tunnel)
> + mibs+=(ucd-snmp/diskio)
> + mibs+=(ucd-snmp/lmsensorsMib)
> + mibs+=(ucd-snmp/extensible)
> + mibs+=(udp-mib)
> + if $have_rtnetlink; then
> + mibs+=(rmon-mib)
> + else
> + mibs+=(Rmon)
> + mibs+=(Rmon/alarm)
> + fi
> + options+=(--with-mysql)
> + options+=(--with-perl-modules)
> + transports+=(AAL5PVC)
> + transports+=(IPX)
> + transports+=(STD)
> + $is_5_8_or_above && transports+=(UDPshared)
> + [ "$opensslver" = "" ] && options+=("--with-openssl=internal")
> + options+=("--with-default-snmp-version=2")
> + { rpm -q python-setuptools >/dev/null 2>&1 || rpm -q python-devel
> >/dev/null 2>&1 || dpkg-query -l python-setuptools >/dev/null 2>&1; }
> && options+=(--with-python-modules)
> + # Disabled modules because troublesome: Rmon
> ipfwchains/ipfwchains ucd-snmp/lmSensors
> + ;;
> + darwin*|Darwin*)
> + mibs+=(disman/nslookup-mib)
> + mibs+=(host)
> + mibs+=(if-mib)
> + mibs+=(mibII/mta_sendmail)
> + mibs+=(misc/ipfwacc)
> + mibs+=(snmp-notification-mib)
> + mibs+=(snmp-usm-dh-objects-mib)
> + mibs+=(tcp-mib)
> + mibs+=(ucd-snmp/diskio)
> + mibs+=(ucd-snmp/extensible)
> + mibs+=(udp-mib)
> + mibs+=(Rmon)
> + options+=(--with-python-modules)
> + if [ "${TRAVIS_OS_NAME}" == "osx" ]; then
> + options+=("--with-openssl=$(brew --prefix openssl)")
> + # On Travis OS/X linking the Perl modules fails with
> "undefined symbol
> + # _BN_bin2bn" etc.
> + options+=(--disable-embedded-perl --without-perl-modules)
> + fi
> + ;;
> + CYGWIN*)
> + export ENV_SEPARATOR=":"
> + options+=(--disable-embedded-perl)
> + mibs+=(ucd-snmp/extensible)
> + mibs+=(winExtDLL)
> + #options+=("--with-openssl=internal")
> + if [ -n "$APPVEYOR" ]; then
> + # On Appveyor building the Perl modules fails as follows:
> + #
> /usr/lib/perl5/5.26/i686-cygwin-threads-64int/CORE/reentr.h:104:26:
> fatal error: crypt.h: No such file or directory
> + options+=(--disable-embedded-perl --without-perl-modules)
> + fi
> + ;;
> + MINGW*|MSYS*)
> + options+=(--disable-embedded-perl)
> + if [ "$branch_name" != "V5-4-patches" ]; then
> + options+=("--with-openssl=/c/OpenSSL")
> + fi
> + mibs+=(snmp-notification-mib)
> + mibs+=(ucd-snmp/dlmod)
> + mibs+=(ucd-snmp/extensible)
> + mibs+=(winExtDLL)
> + options+=(--without-perl-modules)
> + #options+=(--with-mibdirs="C:/usr/share/snmp/mibs")
> + if [ -n "$APPVEYOR" ]; then
> + options+=(--build=x86_64-pc-mingw32)
> + fi
> + ;;
> + *)
> + echo "Unknown OS $(uname)."
> + ;;
> +esac
> +
> +with_openssl=""
> +for o in "${options[@]}"; do
> + if [ "${o#--with-openssl=}" != "$o" ]; then
> + with_openssl="${o#--with-openssl=}"
> + fi
> +done
> +set -x
> +if [ "$opensslver" != "" ] &&
> + grep -rqw BIO_dgram_get_peer ${with_openssl:-/usr}/include/openssl
> 2>/dev/null
> +then
> + transports+=(DTLSUDP)
> + # to do: figure out why one of the IPv6 TLSTCP test fails on
> Travis.
> + if [ "${TRAVIS_OS_NAME}" != linux ]; then
> + transports+=(TLSTCP)
> + fi
> +fi
> +set +x
> +$is_5_8_or_above && options+=(--enable-blumenthal-aes)
> +security_modules=(usm tsm)
> +if [ -e /usr/include/krb5.h ]; then
> + security_modules+=(ksm)
> +fi
> +set -x
> +./configure "${options[@]}" --with-transports="${transports[*]}" \
> + --with-security-modules="${security_modules[*]}" \
> + --with-mib-modules="${mibs[*]}" "$@"
> +set +x
> diff --git a/ci/net-snmp-run-perl-tests b/ci/net-snmp-run-perl-tests
> new file mode 100755
> index 000000000000..e87839580a11
> --- /dev/null
> +++ b/ci/net-snmp-run-perl-tests
> @@ -0,0 +1,9 @@
> +#!/bin/bash
> +
> +# See also the output of the following command:
> +# find -name '*.so' | sed 's/^\.\///;s/\/[^\/]*$//' | sort -u
> +export
> LD_LIBRARY_PATH="$PWD/
> snmplib/.libs:$PWD/agent/helpers/.libs:$PWD/agent/.libs:$PWD/apps/.libs:"
> &&
> +make perlmakefiles &&
> +cd perl &&
> +make &&
> +make test
> diff --git a/ci/net-snmp-run-python-tests b/ci/net-snmp-run-python-tests
> new file mode 100755
> index 000000000000..df483aa0f27a
> --- /dev/null
> +++ b/ci/net-snmp-run-python-tests
> @@ -0,0 +1,16 @@
> +#!/bin/bash
> +
> +set -x
> +killall snmpd
> +export
> LD_LIBRARY_PATH="$PWD/
> snmplib/.libs:$PWD/agent/.libs:$PWD/agent/helpers/.libs:"
> +export MIBDIRS=$PWD/mibs
> +export SNMP_PERSISTENT_DIR=/tmp/net-snmp
> +export SNMP_SNMPD_PORT=1161
> +export PYTHONPATH=$PWD/python/netsnmp
> +mkdir -p "$SNMP_PERSISTENT_DIR" || exit $?
> +agent/snmpd -I-smux -f -Lo -c python/netsnmp/tests/snmpd.conf
> localhost:${SNMP_SNMPD_PORT} &
> +pid=$!
> +cd python || exit $?
> +python setup.py --basedir="$PWD/.." test || exit $?
> +kill $pid
> +wait
> diff --git a/ci/net-snmp-run-tests b/ci/net-snmp-run-tests
> new file mode 100755
> index 000000000000..29fd62491d18
> --- /dev/null
> +++ b/ci/net-snmp-run-tests
> @@ -0,0 +1,37 @@
> +#!/bin/bash
> +
> +scriptdir="$(dirname "$0")"
> +
> +killall snmpd snmptrapd >&/dev/null
> +
> +export SNMP_VERBOSE=1 &&
> +export SNMP_SAVE_TMPDIR=1 &&
> +export NETSNMP_DNS_WORKAROUND=1 &&
> +make -s &&
> +(
> + export PATH="$PWD/agent/.libs:$PWD/apps/.libs:$PWD:$PATH" &&
> + export
> LD_LIBRARY_PATH="$PWD/
> snmplib/.libs:$PWD/agent/.libs:$PWD/agent/helpers/.libs:$PWD/perl/blib/arch/auto/NetSNMP/default_store:$PWD/perl/blib/arch/auto/SNMP:$PWD/perl/blib/arch/auto/NetSNMP/ASN:$PWD/perl/blib/arch/auto/NetSNMP/OID:$PWD/perl/blib/arch/auto/NetSNMP/TrapReceiver:$PWD/perl/blib/arch/auto/NetSNMP/agent:$PWD/perl/blib/arch/auto/NetSNMP/agent/default_store:"
> &&
> + export
> PERL5LIB="$PWD/perl/blib/
> lib:$PWD/perl/blib/lib/NetSNMP:$PWD/perl/blib/lib/NetSNMP/agent"
> &&
> + cd testing &&
> + if [ -e RUNFULLTESTS ] &&
> + perl -e 'require TAP::Harness;' >/dev/null 2>&1; then
> + if [ "${TRAVIS_OS_NAME}" == "osx" ]; then
> + for g in default snmpv3 unit-tests; do
> + echo "==== Test group $g ===="
> + ./RUNFULLTESTS -g $g || exit $?
> + echo "==== End of test group $g ===="
> + done
> + else
> + ./RUNFULLTESTS -g all
> + fi
> + else
> + make -s test
> + fi &&
> + cd .. &&
> + if [ "${TRAVIS_OS_NAME}" != "osx" ]; then
> + "$scriptdir/net-snmp-run-perl-tests"
> + fi
> +) &&
> +if [ "${TRAVIS_OS_NAME}" != "osx" ]; then
> + "$scriptdir/net-snmp-run-python-tests"
> +fi
> diff --git a/testing/fulltests/default/T200snmpv2cwalkall_simple
> b/testing/fulltests/default/T200snmpv2cwalkall_simple
> index 5d5f16f7dc63..82bfd4785c4c 100644
> --- a/testing/fulltests/default/T200snmpv2cwalkall_simple
> +++ b/testing/fulltests/default/T200snmpv2cwalkall_simple
> @@ -4,6 +4,8 @@
>
> HEADER "full snmpwalk (SNMPv2c) against agent (may take time)"
>
> +[ "${TRAVIS_OS_NAME}" = "osx" ] && SKIP Skipping MIB walk on Travis OS/X
> +
> if test `uname -s` = "HP-UX" ; then
> if test `id -u` != "0" ; then
> # The agent needs to be run as root - else force skip
> diff --git a/testing/fulltests/support/simple_eval_tools.sh
> b/testing/fulltests/support/simple_eval_tools.sh
> index 4d3beb444c6f..fdd7791d9a3a 100644
> --- a/testing/fulltests/support/simple_eval_tools.sh
> +++ b/testing/fulltests/support/simple_eval_tools.sh
> @@ -1,3 +1,4 @@
> +#!/bin/sh
> #
> # eval_tools.sh
> #
> @@ -47,10 +48,10 @@ fi
> #
> HEADER() {
> if [ "x$SNMP_HEADERONLY" != "x" ]; then
> - echo test $*
> + echo "test $*"
> exit 0;
> else
> - { echo "# testing $*"; echo ""; } >> $SNMP_TMPDIR/invoked
> + { echo "# testing $*"; echo ""; } >> "$SNMP_TMPDIR/invoked"
> fi
> }
>
> @@ -68,7 +69,7 @@ GRONK
> }
>
> CAN_USLEEP() {
> - if [ "$SNMP_CAN_USLEEP" = 0 -o "$SNMP_CAN_USLEEP" = 1 ] ; then
> + if [ "$SNMP_CAN_USLEEP" = 0 ] || [ "$SNMP_CAN_USLEEP" = 1 ] ; then
> return 0
> fi
> sleep .1 > /dev/null 2>&1
> @@ -133,7 +134,7 @@ SKIPIF() {
> VERIFY() { # <path_to_file(s)>
> missingfiles=""
>
> - for f in $*; do
> + for f in "$@"; do
> [ -f "$f" ] && continue
> echo "FAILED: Cannot find file \"$f\"."
> missingfiles=true
> @@ -196,14 +197,14 @@ CAPTURE() { # <command_with_arguments_to_execute>
> # track invoked command per test when verbose
> if [ $SNMP_VERBOSE -gt 0 ]; then
> OUTPUTENVVARS $junkoutputfile.invoked
> - echo $* >> $junkoutputfile.invoked
> + echo "$@" >> $junkoutputfile.invoked
> fi
>
> if [ $loggedvars = 0 ]; then
> OUTPUTENVVARS $SNMP_TMPDIR/invoked
> loggedvars=1
> fi
> - echo $* >> $SNMP_TMPDIR/invoked
> + echo "$@" >> $SNMP_TMPDIR/invoked
>
> if [ $SNMP_VERBOSE -gt 0 ]; then
> cat <<KNORG
> @@ -409,7 +410,7 @@ WAITFORNOTCOND() {
>
> # Wait until the shell statement "$@" evaluates to true.
> WAITFORCOND() {
> - WAITFORNOTCOND if "$@;" then false ";" else true ";" fi
> + WAITFORNOTCOND "if $@; then false; else true; fi"
> }
>
> WAITFORAGENT() {
> @@ -685,6 +686,12 @@ FINISHED() {
> rm -f core
> fi
> echo "$headerStr...FAIL" >> $SNMP_TMPDIR/invoked
> + {
> + find "$SNMP_TMPDIR" -type f |
> + while read -r f; do
> + head -n 256 "$f"; tail -n 256 "$f";
> + done;
> + } 1>&2
> exit 1
> fi
>
> --
> 2.16.3
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Net-snmp-coders mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/net-snmp-coders
--
Thanks,
Keith (pantherse)
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot