[PATCH, RFC] Add Travis and Appveyor CI support
Bart Van Assche <[email protected]>
| Newsgroups | gmane.network.net-snmp.devel |
|---|---|
| Message-ID | <[email protected]> |
This patch makes it possible to enable Travis and Appveyor continuous
integration support. On Travis Net-SNMP is built with all MIBs, with
--disable-set-support, with --enable-mini-agent, with --enable-minimalist
and with --enable-read-only. All these configurations are built on Linux
and on OS/X. On Appveyor Net-SNMP is built with MSVC and dynamic
libraries, with MSVC and statically linked libraryes, with MinGW64, with
Cygwin32 and with Cygwin64. As one can see in the scripts in the "ci"
directory the to-do items are as follows:
- Fix the MinGW64 build. That build fails with a linker error.
- Make the regression tests pass for the Cygwin builds.
- Make the regression tests pass for --disable-set-support,s
--enable-mini-agent, --enable-minimalist and --enable-read-only.
- Make the non-default MIBs compatible with --disable-set-support,
--enable-mini-agent, --enable-minimalist and --enable-read-only.
For the current build output, see also:
- https://travis-ci.org/bvanassche/net-snmp/
- https://ci.appveyor.com/project/BVanAssche87257/net-snmp
---
.appveyor.yml | 20 ++
.travis.yml | 39 ++++
ci/build.bat | 43 ++++
ci/build.sh | 21 ++
ci/net-snmp-configure | 252 +++++++++++++++++++++
ci/net-snmp-run-perl-tests | 9 +
ci/net-snmp-run-python-tests | 16 ++
ci/net-snmp-run-tests | 37 +++
ci/openssl.bat | 3 +
.../fulltests/default/T200snmpv2cwalkall_simple | 2 +
10 files changed, 442 insertions(+)
create mode 100644 .appveyor.yml
create mode 100644 .travis.yml
create mode 100644 ci/build.bat
create mode 100755 ci/build.sh
create mode 100755 ci/net-snmp-configure
create mode 100755 ci/net-snmp-run-perl-tests
create mode 100755 ci/net-snmp-run-python-tests
create mode 100755 ci/net-snmp-run-tests
create mode 100644 ci/openssl.bat
diff --git a/.appveyor.yml b/.appveyor.yml
new file mode 100644
index 000000000000..1a8fd5f64844
--- /dev/null
+++ b/.appveyor.yml
@@ -0,0 +1,20 @@
+image:
+ - Visual Studio 2017
+
+environment:
+ CYG_MIRROR: http://mirrors.kernel.org/sourceware/cygwin/
+ matrix:
+ - BUILD: MSVCDYNAMIC64
+ - BUILD: MSVCSTATIC64
+ - BUILD: MinGW64
+ - BUILD: Cygwin32
+ - BUILD: Cygwin64
+
+matrix:
+ allow_failures:
+ - BUILD: MINGW64
+
+clone_depth: 5
+
+build_script:
+ - 'call "ci\build.bat"'
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 000000000000..dc5c7d9a213f
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,39 @@
+language: c
+
+os:
+ - linux
+ - osx
+
+env:
+ - env: MODE=regular
+ - env: MODE=disable-set
+ - env: MODE=mini
+ - env: MODE=minimalist
+ - env: MODE=read-only
+
+addons:
+ apt:
+ packages:
+ - dpkg
+ - gcc
+ - libatm-dev
+ - libperl-dev
+ - libsensors4-dev
+ - libssh-dev
+ - libssl-dev
+ - make
+ - perl-modules
+ - pkg-config
+ - python-dev
+ - python-setuptools
+
+sudo: required
+
+before_script:
+ # Add an IPv6 config - see the corresponding Travis issue
+ # https://github.com/travis-ci/travis-ci/issues/8361
+ - 'if [ "${TRAVIS_OS_NAME}" == "linux" ]; then sudo sh -c ''echo 0 > /proc/sys/net/ipv6/conf/all/disable_ipv6; printf "\n::1 localhost ipv6-localhost ipv6-loopback\n" >>/etc/hosts; cat /etc/hosts''; fi'
+ # Upgrade openssl such that Net-SNMP can be built with Blumenthal AES support.
+ - 'if [ "${TRAVIS_OS_NAME}" == "osx" ]; then brew upgrade openssl >/dev/null 2>&1; fi'
+
+script: ci/build.sh
diff --git a/ci/build.bat b/ci/build.bat
new file mode 100644
index 000000000000..6faf1c6560db
--- /dev/null
+++ b/ci/build.bat
@@ -0,0 +1,43 @@
+echo "Build type %BUILD%"
+@echo on
+goto %BUILD%
+echo "Error: unknown build type %BUILD%"
+goto eof
+
+:MSVCDYNAMIC64
+call "ci\openssl.bat"
+call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
+cd win32
+perl Configure --config=release --with-sdk --with-ipv6 --with-winextdll --linktype=dynamic --with-ssl --with-sslincdir=C:\OpenSSL-Win64\include --with-ssllibdir=C:\OpenSSL-Win64\lib\vc
+nmake
+goto eof
+
+:MSVCSTATIC64
+call "ci\openssl.bat"
+call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\vcvarsall.bat" amd64
+cd win32
+perl Configure --config=release --with-sdk --with-ipv6 --with-winextdll --linktype=static --with-ssl --with-sslincdir=C:\OpenSSL-Win64\include --with-ssllibdir=C:\OpenSSL-Win64\lib\vc
+
+nmake
+goto eof
+
+:MinGW32
+REM to do - install MinGW32 first.
+C:\mingw\msys\1.0\usr\bin\bash --login -c 'set -x; cd "${APPVEYOR_BUILD_FOLDER}"; ci/build.sh'
+goto eof
+
+:MinGW64
+C:\msys64\usr\bin\bash --login -c 'set -x; cd "${APPVEYOR_BUILD_FOLDER}"; ci/build.sh'
+goto eof
+
+:Cygwin32
+c:\cygwin\setup-x86.exe --quiet-mode --no-shortcuts --only-site --site "%CYG_MIRROR%" --packages openssl-devel > NUL
+c:\cygwin\bin\bash --login -c 'set -x; cd "${APPVEYOR_BUILD_FOLDER}"; ci/build.sh'
+goto eof
+
+:Cygwin64
+c:\cygwin64\setup-x86_64.exe --quiet-mode --no-shortcuts --only-site --site "%CYG_MIRROR%" --packages openssl-devel > NUL
+c:\cygwin64\bin\bash --login -c 'set -x; cd "${APPVEYOR_BUILD_FOLDER}"; ci/build.sh'
+goto eof
+
+:eof
diff --git a/ci/build.sh b/ci/build.sh
new file mode 100755
index 000000000000..555026cbce10
--- /dev/null
+++ b/ci/build.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+scriptdir="$(dirname "$0")"
+export NOAUTODEPS=1
+export SNMP_VERBOSE=1
+if [ -z "$OSTYPE" ]; then
+ case "$(uname)" in
+ Linux) OSTYPE=linux;;
+ Darwin) OSTYPE=darwin;;
+ *) OSTYPE="UNKNOWN:$(uname)";;
+ esac
+ export OSTYPE
+fi
+"${scriptdir}"/net-snmp-configure master || exit $?
+make -s || exit $?
+[ "$OSTYPE" = cygwin ] && exit 0
+case "$MODE" in
+ disable-set|mini*|read-only)
+ exit 0;;
+esac
+"${scriptdir}"/net-snmp-run-tests || exit $?
diff --git a/ci/net-snmp-configure b/ci/net-snmp-configure
new file mode 100755
index 000000000000..18b56088d403
--- /dev/null
+++ b/ci/net-snmp-configure
@@ -0,0 +1,252 @@
+#!/usr/bin/env bash
+
+# Necessary software packages for building Net-SNMP:
+# autoconf automake libtool m4
+# glibc glibc-devel libssh2 libssh2-devel openssl openssl-devel rpm-devel
+# perl-ExtUtils-MakeMaker perl-ExtUtils-Embed perl-Test-Harness
+# python python-devel python-setuptools
+
+branch_name="$1"
+shift
+
+is_5_7_or_above="false"
+is_5_8_or_above="false"
+if [ "${branch_name#master}" != "${branch_name}" ]; then
+ is_5_8_or_above="true"
+ is_5_7_or_above="true"
+elif [ "${branch_name#V5-7-}" != "${branch_name}" ]; then
+ is_5_7_or_above="true"
+fi
+have_rtnetlink=true
+set -x
+if ! echo "$*" | grep -Eq 'with-openssl=(no|internal)'; then
+ case "${OSTYPE}" in
+ linux*)
+ opensslver="$(
+ for p in openssl-devel libopenssl-devel libressl-devel; do
+ rpm -q --qf '%{version}\n' $p 2>/dev/null;
+ done;
+ dpkg-query --show -f '${Version}\n' libssl-dev 2>/dev/null;
+ pacman -Q openssl 2>&1 | sed -n 's/^openssl[[:blank:]]*//p'
+ )";;
+ *)
+ opensslver=1.0;;
+ esac
+fi
+set +x
+if grep -q 'CentOS release 4' /etc/issue 2>/dev/null; then
+ opensslver=""
+ have_rtnetlink=false
+fi
+opensshver="$({
+ rpm -q openssh-devel;
+ rpm -q libssh2-devel;
+ rpm -q libssh-devel; } 2>/dev/null |
+ sed -n 's/^.*ssh[0-9]*-devel-\([0-9a-z.]*\)-.*$/\1/p' | head -n1)"
+
+options=()
+options+=(--enable-developer)
+options+=(--enable-ipv6)
+options+=("--prefix=/usr/local/net-snmp-${branch_name}")
+options+=("--with-cflags=$cflags")
+options+=(--with-defaults)
+transports=()
+[ "$opensshver" != "" ] && transports+=(SSH)
+# Mib names can be found as follows:
+# (cd agent/mibgroup &&
+# git grep -lE 'void[[:blank:]]*init_.*\(void\)|config_require\(' |
+# sed -n 's/\.h$//p')
+mibs=()
+mibs+=(mibII)
+case "$MODE" in
+ read-only)
+ ;;
+ *)
+ mibs+=(examples/data_set)
+ mibs+=(examples/delayed_instance)
+ mibs+=(examples/example)
+ mibs+=(examples/notification)
+ mibs+=(examples/scalar_int)
+ mibs+=(examples/ucdDemoPublic)
+ mibs+=(examples/watched)
+ mibs+=(smux)
+ ;;
+esac
+
+case "$(uname)" in
+ Linux*)
+ options+=(--enable-new-features)
+ case "$MODE" in
+ mini*|read-only)
+ ;;
+ *)
+ $is_5_7_or_above && mibs+=(deliver/deliverByNotify)
+ mibs+=(disman/event)
+ #mibs+=(disman/expression) - to do: figure out why this fails on Travis
+ mibs+=(disman/mteEventNotificationTable)
+ mibs+=(disman/mteEventTable)
+ mibs+=(disman/mteObjectsTable)
+ mibs+=(disman/mteTriggerBooleanTable)
+ mibs+=(disman/mteTriggerDeltaTable)
+ mibs+=(disman/mteTriggerExistenceTable)
+ mibs+=(disman/mteTriggerTable)
+ mibs+=(disman/mteTriggerThresholdTable)
+ mibs+=(disman/nslookup-mib)
+ mibs+=(disman/ping-mib)
+ mibs+=(disman/schedule)
+ mibs+=(disman/traceroute-mib)
+ $have_rtnetlink && mibs+=(etherlike-mib)
+ mibs+=(examples/netSnmpHostsTable)
+ mibs+=(hardware/cpu)
+ mibs+=(hardware/fsys)
+ mibs+=(hardware/memory)
+ mibs+=(hardware/sensors)
+ mibs+=(host)
+ mibs+=(ip-forward-mib)
+ mibs+=(ip-mib/inetNetToMediaTable)
+ mibs+=(ip-mib/ipDefaultRouterTable)
+ mibs+=(ip-mib/ipv4InterfaceTable)
+ mibs+=(ip-mib/ipv6InterfaceTable)
+ mibs+=(ip-mib/ipv6ScopeZoneIndexTable)
+ mibs+=(mibII/ipAddr)
+ mibs+=(mibII/mta_sendmail)
+ mibs+=(misc/ipfwacc)
+ mibs+=(sctp-mib)
+ mibs+=(snmp-notification-mib)
+ [ "$opensslver" != "" ] && mibs+=(snmp-usm-dh-objects-mib)
+ mibs+=(tcp-mib)
+ mibs+=(testhandler)
+ [ "$opensslver" != "" ] && mibs+=(tlstm-mib tsm-mib)
+ mibs+=(tunnel)
+ mibs+=(ucd-snmp/diskio)
+ mibs+=(ucd-snmp/lmsensorsMib)
+ mibs+=(ucd-snmp/extensible)
+ mibs+=(udp-mib)
+ if $have_rtnetlink; then
+ mibs+=(rmon-mib)
+ else
+ mibs+=(Rmon)
+ mibs+=(Rmon/alarm)
+ fi
+ ;;
+ esac
+ options+=(--with-mysql)
+ options+=(--with-perl-modules)
+ transports+=(AAL5PVC)
+ transports+=(IPX)
+ transports+=(STD)
+ $is_5_8_or_above && transports+=(UDPshared)
+ [ "$opensslver" = "" ] && options+=("--with-openssl=internal")
+ options+=("--with-default-snmp-version=2")
+ { rpm -q python-setuptools >/dev/null 2>&1 || rpm -q python-devel >/dev/null 2>&1 || dpkg-query -l python-setuptools >/dev/null 2>&1; } && options+=(--with-python-modules)
+ # Disabled modules because troublesome: Rmon ipfwchains/ipfwchains ucd-snmp/lmSensors
+ ;;
+ darwin*|Darwin*)
+ case "$MODE" in
+ mini*|read-only)
+ ;;
+ *)
+ mibs+=(disman/nslookup-mib)
+ mibs+=(host)
+ mibs+=(if-mib)
+ mibs+=(mibII/mta_sendmail)
+ mibs+=(misc/ipfwacc)
+ mibs+=(snmp-notification-mib)
+ mibs+=(snmp-usm-dh-objects-mib)
+ mibs+=(tcp-mib)
+ mibs+=(ucd-snmp/diskio)
+ mibs+=(ucd-snmp/extensible)
+ mibs+=(udp-mib)
+ mibs+=(Rmon)
+ ;;
+ esac
+ options+=(--with-python-modules)
+ if [ "${TRAVIS_OS_NAME}" == "osx" ]; then
+ options+=("--with-openssl=$(brew --prefix openssl)")
+ # On Travis OS/X linking the Perl modules fails with "undefined symbol
+ # _BN_bin2bn" etc.
+ options+=(--disable-embedded-perl --without-perl-modules)
+ fi
+ ;;
+ CYGWIN*)
+ export ENV_SEPARATOR=":"
+ mibs+=(ucd-snmp/extensible)
+ mibs+=(winExtDLL)
+ # To do: avoid that the Net-SNMP headers trigger the following build
+ # failure: /usr/include/w32api/psdk_inc/_fd_types.h:100:2: warning: #warning "fd_set and associated macros have been defined in sys/types. This can cause runtime problems with W32 sockets" [-Wcpp]
+ # #warning "fd_set and associated macros have been defined in sys/types.
+ options+=("--with-openssl=internal")
+ if [ -n "$APPVEYOR" ]; then
+ # On Appveyor building the Perl modules fails as follows:
+ # /usr/lib/perl5/5.26/i686-cygwin-threads-64int/CORE/reentr.h:104:26: fatal error: crypt.h: No such file or directory
+ options+=(--disable-embedded-perl --without-perl-modules)
+ fi
+ ;;
+ MINGW*|MSYS*)
+ options+=(--disable-embedded-perl)
+ if [ "$branch_name" != "V5-4-patches" ]; then
+ options+=("--with-openssl=/c/OpenSSL")
+ fi
+ mibs+=(snmp-notification-mib)
+ mibs+=(ucd-snmp/dlmod)
+ mibs+=(ucd-snmp/extensible)
+ mibs+=(winExtDLL)
+ options+=(--without-perl-modules)
+ #options+=(--with-mibdirs="C:/usr/share/snmp/mibs")
+ if [ -n "$APPVEYOR" ]; then
+ options+=("--build=x86_64-pc-mingw32")
+ fi
+ ;;
+ *)
+ echo "Unknown OS $(uname)."
+ ;;
+esac
+
+with_openssl=""
+for o in "${options[@]}"; do
+ if [ "${o#--with-openssl=}" != "$o" ]; then
+ with_openssl="${o#--with-openssl=}"
+ fi
+done
+set -x
+if [ "$opensslver" != "" ] &&
+ grep -rqw BIO_dgram_get_peer "${with_openssl:-/usr}/include/openssl" 2>/dev/null
+then
+ transports+=(DTLSUDP)
+ # to do: figure out why one of the IPv6 TLSTCP test fails on Travis.
+ if [ "${TRAVIS_OS_NAME}" != linux ]; then
+ transports+=(TLSTCP)
+ fi
+fi
+set +x
+if $is_5_8_or_above; then
+ [ -e "${with_openssl:-/usr}/openssl/aes.h" ] &&
+ [ -e "${with_openssl:-/usr}/openssl/evp.h" ] &&
+ options+=(--enable-blumenthal-aes)
+fi
+security_modules=(usm tsm)
+if [ -e /usr/include/krb5.h ]; then
+ security_modules+=(ksm)
+fi
+
+case "$MODE" in
+ disable-set)
+ options+=(--disable-set-support);;
+ mini)
+ options+=(--enable-mini-agent);;
+ minimalist)
+ options+=(--enable-minimalist);;
+ read-only)
+ options+=(--enable-read-only);;
+ regular|"")
+ ;;
+ *)
+ echo "Error: unknown build type $MODE"
+ exit 1;;
+esac
+
+set -x
+./configure "${options[@]}" --with-transports="${transports[*]}" \
+ --with-security-modules="${security_modules[*]}" \
+ --with-mib-modules="${mibs[*]}" "$@"
+set +x
diff --git a/ci/net-snmp-run-perl-tests b/ci/net-snmp-run-perl-tests
new file mode 100755
index 000000000000..e87839580a11
--- /dev/null
+++ b/ci/net-snmp-run-perl-tests
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+# See also the output of the following command:
+# find -name '*.so' | sed 's/^\.\///;s/\/[^\/]*$//' | sort -u
+export LD_LIBRARY_PATH="$PWD/snmplib/.libs:$PWD/agent/helpers/.libs:$PWD/agent/.libs:$PWD/apps/.libs:" &&
+make perlmakefiles &&
+cd perl &&
+make &&
+make test
diff --git a/ci/net-snmp-run-python-tests b/ci/net-snmp-run-python-tests
new file mode 100755
index 000000000000..96ef35c0affd
--- /dev/null
+++ b/ci/net-snmp-run-python-tests
@@ -0,0 +1,16 @@
+#!/bin/bash
+
+set -x
+killall snmpd
+export LD_LIBRARY_PATH="$PWD/snmplib/.libs:$PWD/agent/.libs:$PWD/agent/helpers/.libs:"
+export MIBDIRS=$PWD/mibs
+export SNMP_PERSISTENT_DIR=/tmp/net-snmp
+export SNMP_SNMPD_PORT=1161
+export PYTHONPATH=$PWD/python/netsnmp
+mkdir -p "$SNMP_PERSISTENT_DIR" || exit $?
+agent/snmpd -I-smux -r -f -Lo -c python/netsnmp/tests/snmpd.conf localhost:${SNMP_SNMPD_PORT} &
+pid=$!
+cd python || exit $?
+python setup.py --basedir="$PWD/.." test || exit $?
+kill $pid
+wait
diff --git a/ci/net-snmp-run-tests b/ci/net-snmp-run-tests
new file mode 100755
index 000000000000..29fd62491d18
--- /dev/null
+++ b/ci/net-snmp-run-tests
@@ -0,0 +1,37 @@
+#!/bin/bash
+
+scriptdir="$(dirname "$0")"
+
+killall snmpd snmptrapd >&/dev/null
+
+export SNMP_VERBOSE=1 &&
+export SNMP_SAVE_TMPDIR=1 &&
+export NETSNMP_DNS_WORKAROUND=1 &&
+make -s &&
+(
+ export PATH="$PWD/agent/.libs:$PWD/apps/.libs:$PWD:$PATH" &&
+ export LD_LIBRARY_PATH="$PWD/snmplib/.libs:$PWD/agent/.libs:$PWD/agent/helpers/.libs:$PWD/perl/blib/arch/auto/NetSNMP/default_store:$PWD/perl/blib/arch/auto/SNMP:$PWD/perl/blib/arch/auto/NetSNMP/ASN:$PWD/perl/blib/arch/auto/NetSNMP/OID:$PWD/perl/blib/arch/auto/NetSNMP/TrapReceiver:$PWD/perl/blib/arch/auto/NetSNMP/agent:$PWD/perl/blib/arch/auto/NetSNMP/agent/default_store:" &&
+ export PERL5LIB="$PWD/perl/blib/lib:$PWD/perl/blib/lib/NetSNMP:$PWD/perl/blib/lib/NetSNMP/agent" &&
+ cd testing &&
+ if [ -e RUNFULLTESTS ] &&
+ perl -e 'require TAP::Harness;' >/dev/null 2>&1; then
+ if [ "${TRAVIS_OS_NAME}" == "osx" ]; then
+ for g in default snmpv3 unit-tests; do
+ echo "==== Test group $g ===="
+ ./RUNFULLTESTS -g $g || exit $?
+ echo "==== End of test group $g ===="
+ done
+ else
+ ./RUNFULLTESTS -g all
+ fi
+ else
+ make -s test
+ fi &&
+ cd .. &&
+ if [ "${TRAVIS_OS_NAME}" != "osx" ]; then
+ "$scriptdir/net-snmp-run-perl-tests"
+ fi
+) &&
+if [ "${TRAVIS_OS_NAME}" != "osx" ]; then
+ "$scriptdir/net-snmp-run-python-tests"
+fi
diff --git a/ci/openssl.bat b/ci/openssl.bat
new file mode 100644
index 000000000000..5082370d63ca
--- /dev/null
+++ b/ci/openssl.bat
@@ -0,0 +1,3 @@
+REM Download and install OpenSSL
+curl https://slproweb.com/download/Win64OpenSSL-1_1_0h.exe -o openssl.exe
+.\openssl.exe /suppressmsgboxes /silent /norestart /nocloseapplications
diff --git a/testing/fulltests/default/T200snmpv2cwalkall_simple b/testing/fulltests/default/T200snmpv2cwalkall_simple
index 5d5f16f7dc63..6ffdfe5aeb30 100644
--- a/testing/fulltests/default/T200snmpv2cwalkall_simple
+++ b/testing/fulltests/default/T200snmpv2cwalkall_simple
@@ -4,6 +4,8 @@
HEADER "full snmpwalk (SNMPv2c) against agent (may take time)"
+[ "${TRAVIS_OS_NAME}" = "osx" ] && SKIP Skipping MIB walk on OS/X
+
if test `uname -s` = "HP-UX" ; then
if test `id -u` != "0" ; then
# The agent needs to be run as root - else force skip
--
2.16.3
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot