Re: [Fuego] Use common script to install fuego on host or docker system
<[email protected]> Wed, 9 Jun 2021 03:35:29 +0000
| Newsgroups | dev.linux.lists.fuego |
|---|---|
| Message-ID | <BYAPR13MB2503BEFCD90F3D8C2905E823FD369@BYAPR13MB2503.namprd13.prod.outlook.com> |
Sorry for the slow response. This looks like a nice refactoring. I did a quick look, and saw a few things that might be problems, but I haven't had time yet to do a more thorough review. This refactoring of the install scripts is something I've wanted to do for a while, so I very much appreciate this work. I'll try to take a closer look, and get back to you with feedback, this week. Thanks, -- Tim > -----Original Message----- > From: [email protected] <[email protected]> > Sent: Friday, June 4, 2021 7:44 AM > To: Bird, Tim <[email protected]>; [email protected] > Cc: venkata pyla <[email protected]>; [email protected]; [email protected] > Subject: [Fuego] Use common script to install fuego on host or docker system > > From: venkata pyla <[email protected]> > > Modified the install scripts to use common scripts for both types of > installation, this way it will help to make sure both the environments > are same and if any future modification will have single place of change. > > Now, for installing fuego on docker system the 'install.sh' script will > use common script in Dockefile to install fuego related stuff. > > sameway for installing fuego on host system it can use the same common > script to install fuego related stuff. > > Below are the common scripts that are used in both the installation types > for each debian distribution > * install-stretch.sh > * install-buster.sh > > so finally, > To install fuego on docker system use the below script with options > $ ./install.sh <options> image_name <port> <distribution> <suite> > e.g: ./install.sh debian_stretch 8090 debian stretch > or > ./install.sh debian_buster 8090 debian buster > or > ./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster > > To install fuego on host system use the below script with options > $ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port> > e.g: ./install-fuego.sh -d debian -s stretch -p 8090 > or > ./install-fuego.sh -d debian -s buster -p 8090 > or > ./install-fuego.sh -d debian -s buster -p 8090 --nojenkins > > Signed-off-by: venkata pyla <[email protected]> > --- > Dockerfile | 303 ++--------------- > Dockerfile.nojenkins | 151 --------- > README | 29 +- > entrypoint.sh | 7 + > fuego-host-scripts/docker-build-image.sh | 11 +- > install-buster.sh | 294 ++++++++++++++++ > install-debian.sh | 410 ++++++++++++++--------- > install-fuego.sh | 109 ++++++ > install-stretch.sh | 288 ++++++++++++++++ > install.sh | 12 +- > uninstall-fuego.sh | 47 +++ > 11 files changed, 1057 insertions(+), 604 deletions(-) > delete mode 100644 Dockerfile.nojenkins > create mode 100755 entrypoint.sh > create mode 100755 install-buster.sh > create mode 100755 install-fuego.sh > create mode 100755 install-stretch.sh > create mode 100644 uninstall-fuego.sh > > diff --git a/Dockerfile b/Dockerfile > index 7f63798..ac57cb2 100644 > --- a/Dockerfile > +++ b/Dockerfile > @@ -2,9 +2,11 @@ > # WARNING: this Dockerfile assumes that the container will be created with > # several bind mounts (see docker-create-container.sh) > # ============================================================================== > -# FIXTHIS: build this as an extension of the nonjenkins image > > -FROM debian:stretch-slim > +ARG SUITE=stretch > +ARG DIST=debian > + > +FROM ${DIST}:${SUITE}-slim > MAINTAINER [email protected] > > # ============================================================================== > @@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY} > ENV https_proxy ${HTTP_PROXY} > > # ============================================================================== > -# Prepare basic image > +# Install fuego with the script > # ============================================================================== > > -ARG DEBIAN_FRONTEND=noninteractive > - > -WORKDIR / > -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list > -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi > - > -# Fuego python dependencies > -# - python-lxml: ftc, loggen > -# - python-simplejson: ftc > -# - python-yaml: ftc > -# - python-openpyxl: ftc (also LTP) > -# - python-requests: ftc (also fuego_release_test) > -# - python-reportlab: ftc > -# - python-parsedatetime: ftc > -# - python-pexpect: ssh_exec (part of ttc) > -# - python-pip: to install filelock, flake8 > -# - filelock: parser > -RUN mkdir -p /usr/share/man/man1 > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \ > - python-lxml python-simplejson python-yaml python-openpyxl \ > - python-requests python-reportlab python-parsedatetime \ > - python-pexpect python-pip python-setuptools python-wheel > -RUN pip install filelock > - > -# Fuego command dependencies > -RUN apt-get -q=2 -V --no-install-recommends install \ > - git sshpass openssh-client sudo net-tools wget curl lava-tool \ > - bash-completion iproute2 > - > -# Default SDK for testing locally or on an x86 board > -RUN apt-get -q=2 -V --no-install-recommends install \ > - build-essential cmake bison flex automake kmod libtool \ > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \ > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools > - > -# Default test host dependencies > -# - iperf iperf3 netperf: used as servers on the host > -# - bzip2 bc: used for local tests by Functional.bzip2/bc > -# - python-matplotlib: Benchmark.iperf3 parser > -# - python-xmltodict: AGL tests > -# - flake8: Functional.fuego_lint > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server) > -# - iputils-ping - for /bin/ping command, used by some tests > -# FIXTHIS: install dependencies dynamically on the tests that need them > -RUN apt-get -q=2 -V --no-install-recommends install \ > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \ > - netpipe-tcp iputils-ping > -RUN pip install flake8 > - > -# miscelaneous packages: > -# python-serial - used by serio > -# diffstat and vim - used by Tim > -# time - useful for timing command duration > -RUN apt-get -q=2 -V --no-install-recommends install \ > - python-serial \ > - diffstat \ > - vim \ > - time > - > -# install packages used by NuttX SDK > -RUN apt-get -q=2 -V --no-install-recommends install \ > - genromfs > - > -# FIXTHIS: determine if these tools are really necessary > -#RUN apt-get -q=2 -V --no-install-recommends install \ > -# apt-utils python-paramiko \ > -# xmlstarlet rsync \ > -# inotify-tools gettext netpipe-tcp \ > -# at minicom lzop bsdmainutils \ > -# mc netcat openssh-server > - > -# bsdmainutils provides hexdump, which is needed for netperf test > -RUN apt-get -q=2 -V --no-install-recommends install \ > - bsdmainutils > - > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash' > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi > - > -# ============================================================================== > -# Install Jenkins with the same UID/GID as the host user > -# ============================================================================== > +RUN mkdir /fuego > +COPY . /fuego/ > +WORKDIR /fuego > > -ARG user=jenkins > -ARG group=jenkins > -ARG uid=1000 > -ARG gid=${uid} > ARG JENKINS_PORT=8090 > -# see https://updates.jenkins.io/download/war/ for latest Jenkins versions > -#ARG JENKINS_VERSION=2.164.2 > -#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab > -#ARG JENKINS_VERSION=2.235.3 > -#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794 > -ARG JENKINS_VERSION=2.249.3 > -ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2 > - > -ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb > -ARG JENKINS_UC=https://updates.jenkins.io > -ARG REF=/var/lib/jenkins/plugins > +ARG NO_JENKINS= > +ARG SUITE=stretch > +ARG DIST=debian > +ARG uid= > +ARG gid= > ENV JENKINS_HOME=/var/lib/jenkins > ENV JENKINS_PORT=$JENKINS_PORT > +RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid $NO_JENKINS > > -# Jenkins dependencies > -RUN apt-get -q=2 -V --no-install-recommends install \ > - default-jdk daemon psmisc adduser procps unzip > -RUN pip install python-jenkins==1.4.0 > - > -RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment > -RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group} > -RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} > -RUN wget -nv ${JENKINS_URL} > -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - > -# allow Jenkins to start and install plugins, as part of dpkg installation > -RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d > -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb > -RUN rm jenkins_${JENKINS_VERSION}_all.deb > - > -# ============================================================================== > -# get ttc script and helpers > -# ============================================================================== > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc > -RUN /usr/local/src/ttc/install.sh /usr/local/bin > -# as of ttc version 2.0.2, this should not be needed > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc > - > -# ============================================================================== > -# get ebf script and helpers > -# ============================================================================== > -RUN apt-get -q=2 -V --no-install-recommends install jq > -RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \ > - cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \ > - chmod a+x /usr/local/bin/ebf > - > -# ============================================================================== > -# Serial Config > -# ============================================================================== > - > -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio > -COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \ > - frontend-install/0002-Output-data-from-port-during-command-execution.patch \ > - /tmp/ > -RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \ > - patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \ > - chown -R jenkins /usr/local/src/serio ; \ > - cp /usr/local/src/serio/serio /usr/local/bin/ ; \ > - ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \ > - ln -s /usr/local/bin/serio /usr/local/bin/sersh' > - > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp > /usr/local/src/serlogin/serlogin /usr/local/bin' > - > -# ============================================================================== > -# fserver > -# ============================================================================== > - > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \ > - ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server' > - > -# ============================================================================== > -# Jenkins post installation > -# ============================================================================== > - > -RUN source /etc/default/jenkins && \ > - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \ > - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins > - > -RUN source /etc/default/jenkins && \ > - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false - > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \ > - if [ -n "$HTTP_PROXY" ]; then \ > - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \ > - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \ > - JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" - > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \ > - fi && \ > - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins; > - > -RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins > - > -# set up Jenkins plugins > -COPY frontend-install/install-plugins.sh \ > - frontend-install/jenkins-support \ > - frontend-install/clitest \ > - /usr/local/bin/ > - > -# start Jenkins onces to initialize directories? > -RUN service jenkins start && \ > - sleep 30 && \ > - service jenkins stop > - > -# install plugins - these versions are for Jenkins version 2.164.2 > -# Explicitly install script-security v1.68, otherwise > -# v1.74 will automatically be installed as a dependency of > -# the junit plugin. Make sure to install before junit plugin. > -# Explicitly install junit:1.27 > - > -# install other plugins from Jenkins update center > -# NOTE: not sure all of these are needed, but keep list > -# as compatible as possible with 1.2.1 release for now > -# Do NOT change the order of the plugins, unless you know what > -# you are doing. Otherwise, the install-plugins.sh script will > -# install the wrong versions of other plugins based on dependency > -# information in the plugin files. > - > -#RUN /usr/local/bin/install-plugins.sh \ > -# script-security:1.68 \ > -# structs:1.20 \ > -# workflow-step-api:2.22 \ > -# workflow-api:2.40 \ > -# junit:1.27 \ > -# scm-api:2.6.3 \ > -# ant:1.9 \ > -# antisamy-markup-formatter:1.5 \ > -# bouncycastle-api:2.17 \ > -# command-launcher:1.3 \ > -# description-setter:1.10 \ > -# display-url-api:2.3.1 \ > -# external-monitor-job:1.7 \ > -# greenballs:1.15 \ > -# icon-shim:2.0.3 \ > -# javadoc:1.5 \ > -# jdk-tool:1.2 \ > -# ldap:1.20 \ > -# mailer:1.23 \ > -# matrix-auth:2.3 \ > -# matrix-project:1.14 \ > -# pam-auth:1.5 \ > -# pegdown-formatter:1.3 \ > -# windows-slaves:1.4 > - > -# install plugins for latest LTS Jenkins (currently 2.249.3) > -RUN /usr/local/bin/install-plugins.sh \ > - script-security \ > - structs \ > - workflow-step-api \ > - workflow-api \ > - junit \ > - scm-api \ > - ant \ > - antisamy-markup-formatter \ > - bouncycastle-api \ > - command-launcher \ > - description-setter \ > - display-url-api \ > - external-monitor-job \ > - greenballs \ > - icon-shim \ > - javadoc \ > - jdk-tool \ > - mailer \ > - matrix-auth \ > - matrix-project \ > - pam-auth \ > - pegdown-formatter \ > - windows-slaves > +COPY entrypoint.sh /usr/bin/ > > -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi > - > -# jenkins should automatically unzip any plugins in the plugin dir > -# make a symlink for mod.js well after flot is installed > -RUN service jenkins start && sleep 30 && \ > - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \ > - ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js > - > -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs > -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf > - > -COPY frontend-install/config.xml $JENKINS_HOME/config.xml > -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > -RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > - > -RUN chown -R jenkins:jenkins $JENKINS_HOME/ > - > -# ============================================================================== > -# ftc post installation > -# ============================================================================== > - > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/ > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc > -RUN echo ". /etc/bash_completion" >> /root/.bashrc > - > -# ============================================================================== > -# Lava > -# ============================================================================== > - > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin > +# Delete the files that are copied for installation > +RUN rm -rf /fuego > +WORKDIR / > > # ============================================================================== > # Setup startup command > # ============================================================================== > - > # FIXTHIS: when running multiple Fuego containers, or if the host is already > # running the netperf server, netperf will complain because the port is taken > -ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash > +ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash > diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins > deleted file mode 100644 > index 56de98a..0000000 > --- a/Dockerfile.nojenkins > +++ /dev/null > @@ -1,151 +0,0 @@ > -# ============================================================================== > -# WARNING: this Dockerfile assumes that the container will be created with > -# several bind mounts (see docker-create-container.sh) > -# ============================================================================== > -# Usage: > -# host$ vi fuego-ro/conf/fuego.conf > -# jenkins_enabled=0 > -# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins > -# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container > -# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container > -# docker# ftc list-boards > -# docker, bbb, ... > -# docker# ftc list-tests > -# Functional.hello_world, ... > -# docker# ftc run-test -b docker -t Functional.hello_world > -# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1) > - > -FROM debian:stretch-slim > -MAINTAINER [email protected] > - > -# ============================================================================== > -# Proxy variables > -# ============================================================================== > - > -ARG HTTP_PROXY > -ENV http_proxy ${HTTP_PROXY} > -ENV https_proxy ${HTTP_PROXY} > - > -# ============================================================================== > -# Prepare basic image > -# ============================================================================== > - > -ARG DEBIAN_FRONTEND=noninteractive > - > -WORKDIR / > -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list > -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi > - > -# Fuego python dependencies > -# - python-lxml: ftc, loggen > -# - python-simplejson: ftc > -# - python-yaml: ftc > -# - python-openpyxl: ftc (also LTP) > -# - python-requests: ftc (also fuego_release_test) > -# - python-reportlab: ftc > -# - python-parsedatetime: ftc > -# - python-pexpect: ssh_exec (part of ttc) > -# - python-pip: to install filelock, flake8 > -# - filelock: parser > -RUN mkdir -p /usr/share/man/man1 > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \ > - python-lxml python-simplejson python-yaml python-openpyxl \ > - python-requests python-reportlab python-parsedatetime \ > - python-pexpect python-pip python-setuptools python-wheel > -RUN pip install filelock > - > -# Fuego command dependencies > -RUN apt-get -q=2 -V --no-install-recommends install \ > - git sshpass openssh-client sudo net-tools wget curl lava-tool \ > - bash-completion iproute2 > - > -# Default SDK for testing locally or on an x86 board > -RUN apt-get -q=2 -V --no-install-recommends install \ > - build-essential cmake bison flex automake kmod libtool \ > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \ > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools > - > -# Default test host dependencies > -# - iperf iperf3 netperf: used as servers on the host > -# - bzip2 bc: used for local tests by Functional.bzip2/bc > -# - python-matplotlib: Benchmark.iperf3 parser > -# - python-xmltodict: AGL tests > -# - flake8: Functional.fuego_lint > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server) > -# - iputils-ping - for /bin/ping command, used by some tests > -# FIXTHIS: install dependencies dynamically on the tests that need them > -RUN apt-get -q=2 -V --no-install-recommends install \ > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \ > - netpipe-tcp iputils-ping > -RUN pip install flake8 > - > -# miscelaneous packages: > -# python-serial - used by serio > -# diffstat and vim - used by Tim > -# time - useful for timing command duration > -RUN apt-get -q=2 -V --no-install-recommends install \ > - python-serial \ > - diffstat \ > - vim \ > - time > - > -# FIXTHIS: determine if these tools are really necessary > -# RUN apt-get -q=2 -V --no-install-recommends install \ > -# apt-utils python-paramiko \ > -# xmlstarlet rsync \ > -# inotify-tools gettext netpipe-tcp \ > -# at minicom lzop bsdmainutils \ > -# mc netcat openssh-server > - > -# bsdmainutils provides hexdump, which is needed for netperf test > -RUN apt-get -q=2 -V --no-install-recommends install \ > - bsdmainutils > - > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash' > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi > - > -# ============================================================================== > -# get ttc script and helpers > -# ============================================================================== > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc > -RUN /usr/local/src/ttc/install.sh /usr/local/bin > -# as of ttc version 2.0.2, this should not be needed > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc > - > -# ============================================================================== > -# Serial Config > -# ============================================================================== > - > -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s > /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' > - > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin > /usr/local/bin' > - > -# ============================================================================== > -# fserver > -# ============================================================================== > - > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server > /usr/local/bin/start_local_bg_server' > - > -# ============================================================================== > -# ftc post installation > -# ============================================================================== > - > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/ > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc > -RUN echo ". /etc/bash_completion" >> /root/.bashrc > - > -# ============================================================================== > -# Lava > -# ============================================================================== > - > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin > - > -# ============================================================================== > -# Setup startup command > -# ============================================================================== > - > -# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken > -ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash > diff --git a/README b/README > index 882d84e..ce42e1e 100644 > --- a/README > +++ b/README > @@ -44,6 +44,29 @@ You can change the name of the docker image and the port used by Jenkins > by passing the corresponding parameters to install.sh. > ./install.sh fuego-8082 8082 > > +To run fuego on different distribution versions like Debian(stretch, buster, etc.) > + pass appropriate arguments to the install.sh script > + ./install.sh <image name> <Jenkins port> <Distribution name> <suite> > + > +By default install.sh script will install fuego on Debian stretch version > + to install fuego on Debian buster version pass below options > + ./install.sh fuego_buster 8090 debian buster > + > +[Note] It is currently verified on Debian versions (stretch and buster), > +and yet to provide install scripts for other distributions > + > +Installing Fuego to a host system > +------------------------------------------------------------------------------- > +Run install-fuego.sh to install fuego on host system, by default it assume the > +host system is Debian stretch > + ./install-fuego.sh > + > + to install on Debian buster version > + ./install-fuego.sh -d debian -s buster -p 8090 > + > + to install without jenkins pass --nojenkins option to the script > + ./install-fuego.sh --nojenkins > + > Running > ------------------------------------------------------------------------------- > To start your Fuego container, issue the following command top directory: > @@ -85,9 +108,9 @@ Updating the Jenkins version > ------------------------------------------------------------------------------- > > Modify the version and checksum > - $ vi Dockerfile > - ARG JENKINS_VERSION=2.32.1 > - ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087 > + $ vi install-debian.sh > + JENKINS_VERSION=2.32.1 > + JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087 > > Re-build the flot plugin > $ sudo apt-get install maven > diff --git a/entrypoint.sh b/entrypoint.sh > new file mode 100755 > index 0000000..6eac895 > --- /dev/null > +++ b/entrypoint.sh > @@ -0,0 +1,7 @@ > +#!/bin/bash > + > +[ "$1" != "--nojenkins" ] && service jenkins start > + > +#TODO: netperf not present in buster, remove this condition once alternative found > +[ -f /etc/init.d/netperf ] && service netperf start > +iperf3 -V -s -D -f M > diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh > index 5823357..6889c47 100755 > --- a/fuego-host-scripts/docker-build-image.sh > +++ b/fuego-host-scripts/docker-build-image.sh > @@ -2,9 +2,10 @@ > # $1 - name for the docker image (default: fuego) > # $2 - port for jenkins (default: 8090) > # $3 - Dockerfile or Dockerfile.nojenkins > +# $4 - Debian code name on which to install (default: stretch) > # > # Example: > -# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 Dockerfile.nojenkins > +# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 debian stretch --nojenkins > # > if [ "$1" = "--no-cache" ]; then > NO_CACHE=--no-cache > @@ -13,7 +14,10 @@ fi > > DOCKERIMAGE=${1:-fuego} > JENKINS_PORT=${2:-8090} > -DOCKERFILE=${3:-Dockerfile} > +DISTRIBUTION=${3:-debian} > +SUITE=${4:-stretch} > +NO_JENKINS=${5:""} > +DOCKERFILE="Dockerfile" > > if [ "$(id -u)" == "0" ]; then > JENKINS_UID=$(id -u $SUDO_USER) > @@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT" > > sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \ > --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \ > + --build-arg DIST=$DISTRIBUTION \ > + --build-arg SUITE=$SUITE \ > + --build-arg NO_JENKINS=$NO_JENKINS \ > --build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE . > diff --git a/install-buster.sh b/install-buster.sh > new file mode 100755 > index 0000000..f4ce65b > --- /dev/null > +++ b/install-buster.sh > @@ -0,0 +1,294 @@ > +#!/bin/bash > +# 2019 (c) Toshiba corp. <[email protected]> > +# > +# Usage: > +# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>] > +# > +usage(){ > + > +cat <<HERE > +Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>] > + > +Install fuego on the host Debian filesystem. > + > +options: > + --help Show usage help > + --nojenkins Install Fuego without Jenkins > +HERE > +} > + > +if [[ $EUID -ne 0 ]]; then > + echo "Sorry, you need root permissions" > + exit 1 > +fi > + > +nojenkins=0 > + > +POSITIONAL=() > +while [[ $# -gt 0 ]] ; do > + case $1 in > + --nojenkins) > + nojenkins=1 > + shift > + ;; > + --help | -h) > + usage > + exit 0 > + ;; > + *) > + POSITIONAL+=("$1") # save argument for later > + shift > + esac > +done > +set -- "${POSITIONAL[@]}" # restore positional arguments > + > +port=${2:-8090} > + > +# ============================================================================== > +# Install Fuego dependencies > +# ============================================================================== > + > +mkdir -p /usr/share/man/man1 > + > +# netperf is in non-free > +echo deb http://deb.debian.org/debian buster main non-free > /etc/apt/sources.list > +echo deb http://security.debian.org/debian-security buster/updates main >> /etc/apt/sources.list > + > +apt-get update -q=2 > + > +# Fuego python dependencies > +apt-get -q=2 -V --no-install-recommends install \ > + python-lxml python-simplejson python-yaml python-openpyxl \ > + python-requests python-reportlab python-parsedatetime \ > + python-pexpect python-pip python-setuptools python-wheel > +pip install filelock > + > +# Fuego command dependencies > +apt-get -q=2 -V --no-install-recommends install \ > + git sshpass openssh-client sudo net-tools wget curl lava-tool \ > + bash-completion iproute2 > + > +# Default SDK for testing locally or on an x86 board > +apt-get -q=2 -V --no-install-recommends install \ > + build-essential cmake bison flex kmod automake libtool \ > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \ > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools > + > +# Default test host dependencies > +apt-get -q=2 -V --no-install-recommends install \ > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \ > + netpipe-tcp iputils-ping > +pip install flake8 > + > +# miscelaneous packages: > +# python-serial - used by serio > +# diffstat and vim - used by Tim > +# time - useful for timing command duration > +apt-get -q=2 -V --no-install-recommends install \ > + python-serial \ > + diffstat \ > + vim \ > + time > + > +# install packages used by NuttX SDK > +apt-get -q=2 -V --no-install-recommends install \ > + genromfs > + > +# bsdmainutils provides hexdump, which is needed for netperf test > +#apt-get -q=2 -V --no-install-recommends install \ > +# bsdmainutils > + > +echo "dash dash/sh boolean false" | debconf-set-selections > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash > + > +# TODO: check if proxy settings can set outside docker container > +if [ -n "$http_proxy" ]; then > + sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc > + echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment > +fi > + > +# ============================================================================== > +# Install Jenkins > +# ============================================================================== > + > +if [ $nojenkins -eq 0 ]; then > + #JENKINS_VERSION=2.164.2 > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab > + #JENKINS_VERSION=2.249.3 > + #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2 > + JENKINS_VERSION=2.164.3 > + JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53 > + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb > + export JENKINS_UC=https://updates.jenkins.io > + REF=/var/lib/jenkins/plugins > + JENKINS_HOME=/var/lib/jenkins > + JENKINS_PORT=$port > + > + # Jenkins dependencies > + apt-get -q=2 -V --no-install-recommends install \ > + default-jdk daemon psmisc adduser procps unzip > + pip install python-jenkins==1.4.0 > + > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment > + export JENKINS_PORT=$port > + > + groupadd jenkins > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins > + > + wget -nv ${JENKINS_URL} > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - > + > + # allow Jenkins to start and install plugins, as part of dpkg installation > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb > + rm jenkins_${JENKINS_VERSION}_all.deb > + > + # fix for Jenkins Java version check error > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version > \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)" > + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins > + > + source /etc/default/jenkins && \ > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \ > + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins > + > + source /etc/default/jenkins && \ > + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false - > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \ > + if [ -n "$http_proxy" ]; then \ > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \ > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \ > + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" - > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \ > + fi && \ > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins; > + > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins > + > + # set up Jenkins plugins > + cp frontend-install/install-plugins.sh \ > + frontend-install/jenkins-support \ > + frontend-install/clitest \ > + /usr/local/bin/ > + > + mkdir -p $JENKINS_HOME/userContent/docs > + cp frontend-install/config.xml $JENKINS_HOME/config.xml > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > + chown -R jenkins:jenkins $JENKINS_HOME/ > + > + # start and stop jenkins to pre-populate some settings > + service jenkins start && \ > + sleep 30 && \ > + service jenkins stop > + > + # install other plugins from Jenkins update center > + # NOTE: not sure all of these are needed, but keep list > + # compatible with 1.2.1 release for now > + #/usr/local/bin/install-plugins.sh \ > + # ant:1.9 \ > + # antisamy-markup-formatter:1.5 \ > + # bouncycastle-api:2.17 \ > + # command-launcher:1.3 \ > + # description-setter:1.10 \ > + # display-url-api:2.3.1 \ > + # external-monitor-job:1.7 \ > + # greenballs:1.15 \ > + # icon-shim:2.0.3 \ > + # javadoc:1.5 \ > + # jdk-tool:1.2 \ > + # junit:1.27 \ > + # ldap:1.20 \ > + # mailer:1.23 \ > + # matrix-auth:2.3 \ > + # matrix-project:1.14 \ > + # pam-auth:1.4 \ > + # pegdown-formatter:1.3 \ > + # structs:1.17 \ > + # windows-slaves:1.4 > + > + # install plugins for latest LTS Jenkins (currently 2.249.3) > + /usr/local/bin/install-plugins.sh \ > + script-security \ > + structs \ > + workflow-step-api \ > + workflow-api \ > + junit \ > + scm-api \ > + ant \ > + antisamy-markup-formatter \ > + bouncycastle-api \ > + command-launcher \ > + description-setter \ > + display-url-api \ > + external-monitor-job \ > + greenballs \ > + icon-shim \ > + javadoc \ > + jdk-tool \ > + mailer \ > + matrix-auth \ > + matrix-project \ > + pam-auth \ > + pegdown-formatter \ > + windows-slaves > + > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi > + > + # make the mod.js symlink well after flot is installed > + service jenkins start && sleep 30 && \ > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \ > + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js > + > + chown -R jenkins:jenkins $JENKINS_HOME/ > +fi > + > +# ============================================================================== > +# get ttc script and helpers > +# ============================================================================== > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc > +/usr/local/src/ttc/install.sh /usr/local/bin > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc > + > +# ============================================================================== > +# Serial Config > +# ============================================================================== > +if [ $nojenkins -eq 0 ]; then > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; > cp /usr/local/src/serlogin/serlogin /usr/local/bin' > +else > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln - > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin > /usr/local/bin' > +fi > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \ > + frontend-install/0002-Output-data-from-port-during-command-execution.patch \ > + /tmp/ > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \ > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;' > + > + > +# ============================================================================== > +# get ebf script and helpers > +# ============================================================================== > +apt-get -q=2 -V --no-install-recommends install jq > +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf > +chmod a+x /usr/local/bin/ebf > + > +# ============================================================================== > +# fserver > +# ============================================================================== > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver > +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server > + > +# ============================================================================== > +# ftc post installation > +# ============================================================================== > +ln -s /fuego-core/scripts/ftc /usr/local/bin/ > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc > +echo ". /etc/bash_completion" >> /root/.bashrc > + > +# ============================================================================== > +# Lava > +# ============================================================================== > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin > diff --git a/install-debian.sh b/install-debian.sh > index ccca032..2d3ffd3 100755 > --- a/install-debian.sh > +++ b/install-debian.sh > @@ -2,102 +2,163 @@ > # 2019 (c) Toshiba corp. <[email protected]> > # > # Usage: > -# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>] > +# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins] > # > -if [ -n "$1" ]; then > - if [ "$1" = "--help" -o "$1" = "-h" ]; then > - cat <<HERE > -Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>] > - > -Installs fuego on the host Debian filesystem. > - > -options: > - --help Show usage help > - --nojenkins Install Fuego without Jenkins > -HERE > - exit 0 > - fi > -fi > + > +usage(){ > + echo "" > + echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]" > + echo "Install fuego on Debian filesystem." > + echo "" > + echo "options:" > + echo " --help | -h Show usage help" > + echo " --suite | -s <suite> Specify Debian suite" > + echo " --port | -p <port> Jenkins port" > + echo " --nojenkins Install Fuego without Jenkins" > + echo "" > +} > > if [[ $EUID -ne 0 ]]; then > echo "Sorry, you need root permissions" > exit 1 > fi > > -if [ "$1" = "--nojenkins" ]; then > - nojenkins=1 > +nojenkins=0 > +port=8090 > +suite=stretch > +user=jenkins > +group=jenkins > +uid=1000 > +gid=1000 > + > +while [ "$1" != "" ]; do > + case $1 in > + -s | --suite ) > + shift > + suite=$1 > + ;; > + -p | --port ) > + shift > + port=$1 > + ;; > + -u | --uid ) > + shift > + uid=$1 > + ;; > + -g | --gid ) > + shift > + gid=$1 > + ;; > + --nojenkins ) > + nojenkins=1 > + ;; > + -h | --help ) > + usage > + exit 0 > + ;; > + * ) > + break > + esac > shift > -else > - nojenkins=0 > - port=${1:-8090} > -fi > +done > > # ============================================================================== > # Install Fuego dependencies > # ============================================================================== > > # netperf is in non-free > -echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list > -echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list > - > -apt-get update -q=2 > +echo deb http://deb.debian.org/debian $suite main non-free > /etc/apt/sources.list > +echo deb http://security.debian.org/debian-security $suite/updates main >> /etc/apt/sources.list > > # Fuego python dependencies > -apt-get -q=2 -V --no-install-recommends install \ > +# - python-lxml: ftc, loggen > +# - python-simplejson: ftc > +# - python-yaml: ftc > +# - python-openpyxl: ftc (also LTP) > +# - python-requests: ftc (also fuego_release_test) > +# - python-reportlab: ftc > +# - python-parsedatetime: ftc > +# - python-pexpect: ssh_exec (part of ttc) > +# - python-pip: to install filelock, flake8 > +# - filelock: parser > +mkdir -p /usr/share/man/man1 > +apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \ > python-lxml python-simplejson python-yaml python-openpyxl \ > python-requests python-reportlab python-parsedatetime \ > - python-pip > + python-pexpect python-pip python-setuptools python-wheel > pip install filelock > > # Fuego command dependencies > apt-get -q=2 -V --no-install-recommends install \ > git sshpass openssh-client sudo net-tools wget curl lava-tool \ > - bash-completion > + bash-completion iproute2 > > # Default SDK for testing locally or on an x86 board > apt-get -q=2 -V --no-install-recommends install \ > build-essential cmake bison flex automake kmod libtool \ > libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \ > - libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools > > # Default test host dependencies > +# - iperf iperf3 netperf: used as servers on the host > +# - bzip2 bc: used for local tests by Functional.bzip2/bc > +# - python-matplotlib: Benchmark.iperf3 parser > +# - python-xmltodict: AGL tests > +# - flake8: Functional.fuego_lint > +# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server) > +# - iputils-ping - for /bin/ping command, used by some tests > +# FIXTHIS: install dependencies dynamically on the tests that need them > apt-get -q=2 -V --no-install-recommends install \ > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict > -pip install flake8 bsdmainutils > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \ > + netpipe-tcp iputils-ping > > -echo "dash dash/sh boolean false" | debconf-set-selections > -DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash > -if [ -n "$http_proxy" ]; then > - sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc > - echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment > +#TODO: find alternate package for netperf in buster version > +if [ $suite != "buster" ];then > + apt-get -q=2 -V --no-install-recommends install netperf > fi > +pip install flake8 > > -# ============================================================================== > -# Clone fuego and fuego-core > -# ============================================================================== > -if [ ! -d "/fuego" ]; then > - cd / > - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git > - ln -s /fuego/fuego-ro /fuego-ro > - ln -s /fuego/fuego-rw /fuego-rw > - cd fuego > - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git > - ln -s /fuego/fuego-core /fuego-core > -fi > +# miscelaneous packages: > +# python-serial - used by serio > +# diffstat and vim - used by Tim > +# time - useful for timing command duration > +apt-get -q=2 -V --no-install-recommends install \ > + python-serial \ > + diffstat \ > + vim \ > + time > + > +# install packages used by NuttX SDK > +apt-get -q=2 -V --no-install-recommends install \ > + genromfs > + > +# FIXTHIS: determine if these tools are really necessary > +#RUN apt-get -q=2 -V --no-install-recommends install \ > +# apt-utils python-paramiko \ > +# xmlstarlet rsync \ > +# inotify-tools gettext netpipe-tcp \ > +# at minicom lzop bsdmainutils \ > +# mc netcat openssh-server > + > +# bsdmainutils provides hexdump, which is needed for netperf test > +apt-get -q=2 -V --no-install-recommends install \ > + bsdmainutils > + > +echo "dash dash/sh boolean false" | debconf-set-selections > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash > > # ============================================================================== > # Install Jenkins > # ============================================================================== > - > if [ $nojenkins -eq 0 ]; then > #JENKINS_VERSION=2.164.2 > #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab > JENKINS_VERSION=2.249.3 > JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2 > JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb > - JENKINS_UC=https://updates.jenkins.io > - REF=/var/lib/jenkins/plugins > - JENKINS_HOME=/var/lib/jenkins > + export JENKINS_UC=https://updates.jenkins.io > + export REF=/var/lib/jenkins/plugins > + export JENKINS_HOME=/var/lib/jenkins > JENKINS_PORT=$port > > # Jenkins dependencies > @@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then > pip install python-jenkins==1.4.0 > > echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment > - groupadd jenkins > - useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins > - cd > - wget ${JENKINS_URL} > + export JENKINS_PORT=$port > + > + getent group ${gid} >/dev/null || groupadd -g ${gid} ${group} > + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} > + > + wget -nv ${JENKINS_URL} > echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - > > # allow Jenkins to start and install plugins, as part of dpkg installation > @@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then > dpkg -i jenkins_${JENKINS_VERSION}_all.deb > rm jenkins_${JENKINS_VERSION}_all.deb > > - # update ownership > - chown -R jenkins:jenkins /fuego > - chown -R jenkins:jenkins /fuego-ro > - chown -R jenkins:jenkins /fuego-rw > - chown -R jenkins:jenkins /fuego-core > +fi > + > +# ============================================================================== > +# get ttc script and helpers > +# ============================================================================== > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc > +/usr/local/src/ttc/install.sh /usr/local/bin > +# as of ttc version 2.0.2, this should not be needed > +#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc > + > +# ============================================================================== > +# get ebf script and helpers > +# ============================================================================== > +if [ $nojenkins -eq 0 ]; then > + apt-get -q=2 -V --no-install-recommends install jq > + git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api > + cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf > + chmod a+x /usr/local/bin/ebf > +fi > + > +# ============================================================================== > +# Serial Config > +# ============================================================================== > +git clone https://github.com/frowand/serio.git /usr/local/src/serio > +if [ $nojenkins -eq 0 ]; then > + cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \ > + frontend-install/0002-Output-data-from-port-during-command-execution.patch \ > + /tmp/ > + patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch > + chown -R jenkins /usr/local/src/serio > +fi > +cp /usr/local/src/serio/serio /usr/local/bin/ > +ln -s /usr/local/bin/serio /usr/local/bin/sercp > +ln -s /usr/local/bin/serio /usr/local/bin/sersh > + > +git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin > +[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin > +cp /usr/local/src/serlogin/serlogin /usr/local/bin > + > +# ============================================================================== > +# fserver > +# ============================================================================== > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver > +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server > + > +# ============================================================================== > +# Jenkins post installation > +# ============================================================================== > +if [ $nojenkins -eq 0 ]; then > + > + # Fix jenkins issue, Java version check fails when java version is 11 > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.- > \]*\\\).*\\\".*/\\\1\\\2\/p;')" > + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins > > source /etc/default/jenkins && \ > JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \ > @@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then > > sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins > > - cp /fuego/frontend-install/install-plugins.sh \ > - /fuego/frontend-install/jenkins-support \ > - /fuego/frontend-install/clitest \ > + # set up Jenkins plugins > + cp frontend-install/install-plugins.sh \ > + frontend-install/jenkins-support \ > + frontend-install/clitest \ > /usr/local/bin/ > > - cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml > - ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs > - mkdir $JENKINS_HOME/userContent/docs > - cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf > - jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > - sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > - chown -R jenkins:jenkins $JENKINS_HOME/ > - > # start and stop jenkins to pre-populate some settings > service jenkins start && \ > sleep 30 && \ > service jenkins stop > > + # install plugins - these versions are for Jenkins version 2.164.2 > + # Explicitly install script-security v1.68, otherwise > + # v1.74 will automatically be installed as a dependency of > + # the junit plugin. Make sure to install before junit plugin. > + # Explicitly install junit:1.27 > + > # install other plugins from Jenkins update center > # NOTE: not sure all of these are needed, but keep list > - # compatible with 1.2.1 release for now > - #/usr/local/bin/install-plugins.sh \ > - # ant:1.9 \ > - # antisamy-markup-formatter:1.5 \ > - # bouncycastle-api:2.17 \ > - # command-launcher:1.3 \ > - # description-setter:1.10 \ > - # display-url-api:2.3.1 \ > - # external-monitor-job:1.7 \ > - # greenballs:1.15 \ > - # icon-shim:2.0.3 \ > - # javadoc:1.5 \ > - # jdk-tool:1.2 \ > - # junit:1.27 \ > - # ldap:1.20 \ > - # mailer:1.23 \ > - # matrix-auth:2.3 \ > - # matrix-project:1.14 \ > - # pam-auth:1.4 \ > - # pegdown-formatter:1.3 \ > - # structs:1.17 \ > - # windows-slaves:1.4 > - > - # install plugins for latest LTS Jenkins (currently 2.249.3) > - /usr/local/bin/install-plugins.sh \ > - script-security \ > - structs \ > - workflow-step-api \ > - workflow-api \ > - junit \ > - scm-api \ > - ant \ > - antisamy-markup-formatter \ > - bouncycastle-api \ > - command-launcher \ > - description-setter \ > - display-url-api \ > - external-monitor-job \ > - greenballs \ > - icon-shim \ > - javadoc \ > - jdk-tool \ > - mailer \ > - matrix-auth \ > - matrix-project \ > - pam-auth \ > - pegdown-formatter \ > - windows-slaves > - > - cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi > + # as compatible as possible with 1.2.1 release for now > + # Do NOT change the order of the plugins, unless you know what > + # you are doing. Otherwise, the install-plugins.sh script will > + # install the wrong versions of other plugins based on dependency > + # information in the plugin files. > > + #/usr/local/bin/install-plugins.sh \ > + # script-security:1.68 \ > + # structs:1.20 \ > + # workflow-step-api:2.22 \ > + # workflow-api:2.40 \ > + # junit:1.27 \ > + # scm-api:2.6.3 \ > + # ant:1.9 \ > + # antisamy-markup-formatter:1.5 \ > + # bouncycastle-api:2.17 \ > + # command-launcher:1.3 \ > + # description-setter:1.10 \ > + # display-url-api:2.3.1 \ > + # external-monitor-job:1.7 \ > + # greenballs:1.15 \ > + # icon-shim:2.0.3 \ > + # javadoc:1.5 \ > + # jdk-tool:1.2 \ > + # ldap:1.20 \ > + # mailer:1.23 \ > + # matrix-auth:2.3 \ > + # matrix-project:1.14 \ > + # pam-auth:1.5 \ > + # pegdown-formatter:1.3 \ > + # windows-slaves:1.4 > + > + # install plugins for latest LTS Jenkins (currently 2.249.3) > + /usr/local/bin/install-plugins.sh \ > + script-security \ > + structs \ > + workflow-step-api \ > + workflow-api \ > + junit \ > + scm-api \ > + ant \ > + antisamy-markup-formatter \ > + bouncycastle-api \ > + command-launcher \ > + description-setter \ > + display-url-api \ > + external-monitor-job \ > + greenballs \ > + icon-shim \ > + javadoc \ > + jdk-tool \ > + mailer \ > + matrix-auth \ > + matrix-project \ > + pam-auth \ > + pegdown-formatter \ > + windows-slaves > + > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi > + > + # jenkins should automatically unzip any plugins in the plugin dir > # make the mod.js symlink well after flot is installed > service jenkins start && sleep 30 && \ > rm $JENKINS_HOME/plugins/flot/flot/mod.js && \ > ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js > > - chown -R jenkins:jenkins $JENKINS_HOME/ > -else > - sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf > -fi > + mkdir -p $JENKINS_HOME/userContent/docs > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf > > -# ============================================================================== > -# get ttc script and helpers > -# ============================================================================== > -git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc > -/usr/local/src/ttc/install.sh /usr/local/bin > -perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc > + cp frontend-install/config.xml $JENKINS_HOME/config.xml > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > > -# ============================================================================== > -# Serial Config > -# ============================================================================== > -if [ $nojenkins -eq 0 ]; then > - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; > cp /usr/local/src/serlogin/serlogin /usr/local/bin' > -else > - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln - > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin > /usr/local/bin' > + chown -R jenkins:jenkins $JENKINS_HOME/ > fi > > -# ============================================================================== > -# fserver > -# ============================================================================== > -git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver > -ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server > - > # ============================================================================== > # ftc post installation > # ============================================================================== > ln -s /fuego-core/scripts/ftc /usr/local/bin/ > -cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc > echo ". /etc/bash_completion" >> /root/.bashrc > > # ============================================================================== > @@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc > # ============================================================================== > ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin > ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin > - > -# ============================================================================== > -# Small guide > -# ============================================================================== > -echo "Run 'service netperf start' to start a netperf server" > -echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server" > -echo "Run 'ftc list-boards' to see the available boards" > -echo "Run 'ftc list-tests' to see the available tests" > -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world" > -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone" > -echo "Run 'ftc gen-report' to get results" > diff --git a/install-fuego.sh b/install-fuego.sh > new file mode 100755 > index 0000000..7b3e010 > --- /dev/null > +++ b/install-fuego.sh > @@ -0,0 +1,109 @@ > +#!/bin/bash > +# 2019 (c) Toshiba corp. <[email protected]> > +# > +# Usage: > +# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins] > +# > + > +usage(){ > + echo "" > + echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]" > + echo "Install fuego on Debian filesystem." > + echo "" > + echo "options:" > + echo " --help | -h Show usage help" > + echo " --dist | -d <distribution> Specify distribution" > + echo " --suite | -s <suite> Specify suite" > + echo " --port | -p <port> Jenkins port" > + echo " --nojenkins Install Fuego without Jenkins" > + echo "" > +} > + > +if [[ $EUID -ne 0 ]]; then > + echo "Sorry, you need root permissions" > + exit 1 > +fi > + > +nojenkins= > +port=8090 > +dist=debian > +suite=stretch > +fuego_dir=$(pwd) > + > +while [ "$1" != "" ]; do > + case $1 in > + -d | --dist ) > + shift > + dist=$1 > + ;; > + -s | --suite ) > + shift > + suite=$1 > + ;; > + -p | --port ) > + shift > + port=$1 > + ;; > + --nojenkins ) > + nojenkins="--nojenkins" > + ;; > + -h | --help ) > + usage > + exit 0 > + ;; > + * ) > + break > + esac > + shift > +done > + > +# ============================================================================== > +# Clone fuego-core > +# ============================================================================== > +if [ ! -f "fuego-core/scripts/ftc" ]; then > + # set fuego-core branch to same as current fuego branch > + # get current git branch > + set -o noglob > + while IFS=" " read -r part1 part2 ; do > + if [ $part1 = "*" ] ; then > + branch=$part2 > + fi > + done < <(git branch) > + set +o noglob > + git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git > +fi > + > +# ============================================================================== > +# Install fuego on host system > +# ============================================================================== > +./install-$dist.sh -s $suite -p $port $nojenkins > + > +systemctl daemon-reload > +service jenkins restart > + > +# provide permission to current user > +usermod -aG jenkins $USER > +chmod -R 775 $fuego_dir/fuego-rw > + > +chown -R jenkins:jenkins $fuego_dir/fuego-ro > +chown -R jenkins:jenkins $fuego_dir/fuego-rw > +chown -R jenkins:jenkins $fuego_dir/fuego-core > + > +# ============================================================================== > +# Create Links to fuego folders > +# ============================================================================== > +ln -s $fuego_dir/fuego-ro / > +ln -s $fuego_dir/fuego-rw / > +ln -s $fuego_dir/fuego-core / > + > +# ============================================================================== > +# Small guide > +# ============================================================================== > +echo "Logout and Login current user session to work properly" > +echo "Run 'service netperf start' to start a netperf server" > +echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server" > +echo "Run 'ftc list-boards' to see the available boards" > +echo "Run 'ftc list-tests' to see the available tests" > +echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world" > +echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone" > +echo "Run 'ftc gen-report' to get results" > diff --git a/install-stretch.sh b/install-stretch.sh > new file mode 100755 > index 0000000..4f6f724 > --- /dev/null > +++ b/install-stretch.sh > @@ -0,0 +1,288 @@ > +#!/bin/bash > +# 2019 (c) Toshiba corp. <[email protected]> > +# > +# Usage: > +# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>] > +# > +usage(){ > + > +cat <<HERE > +Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>] > + > +Install fuego on the host Debian filesystem. > + > +options: > + --help Show usage help > + --nojenkins Install Fuego without Jenkins > +HERE > +} > + > +if [[ $EUID -ne 0 ]]; then > + echo "Sorry, you need root permissions" > + exit 1 > +fi > + > +nojenkins=0 > + > +POSITIONAL=() > +while [[ $# -gt 0 ]] ; do > + case $1 in > + --nojenkins) > + nojenkins=1 > + shift > + ;; > + --help | -h) > + usage > + exit 0 > + ;; > + *) > + POSITIONAL+=("$1") # save argument for later > + shift > + esac > +done > +set -- "${POSITIONAL[@]}" # restore positional arguments > + > +port=${2:-8090} > + > +# ============================================================================== > +# Install Fuego dependencies > +# ============================================================================== > + > +mkdir -p /usr/share/man/man1 > + > +# netperf is in non-free > +echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list > +echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list > + > +apt-get update -q=2 > + > +# Fuego python dependencies > +apt-get -q=2 -V --no-install-recommends install \ > + python-lxml python-simplejson python-yaml python-openpyxl \ > + python-requests python-reportlab python-parsedatetime \ > + python-pexpect python-pip python-setuptools python-wheel > +pip install filelock > + > +# Fuego command dependencies > +apt-get -q=2 -V --no-install-recommends install \ > + git sshpass openssh-client sudo net-tools wget curl lava-tool \ > + bash-completion iproute2 > + > +# Default SDK for testing locally or on an x86 board > +apt-get -q=2 -V --no-install-recommends install \ > + build-essential cmake bison flex kmod automake libtool \ > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \ > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools > + > +# Default test host dependencies > +apt-get -q=2 -V --no-install-recommends install \ > + iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \ > + netpipe-tcp iputils-ping > +pip install flake8 > + > +# miscelaneous packages: > +# python-serial - used by serio > +# diffstat and vim - used by Tim > +# time - useful for timing command duration > +apt-get -q=2 -V --no-install-recommends install \ > + python-serial \ > + diffstat \ > + vim \ > + time > + > +# install packages used by NuttX SDK > +apt-get -q=2 -V --no-install-recommends install \ > + genromfs > + > +# bsdmainutils provides hexdump, which is needed for netperf test > +apt-get -q=2 -V --no-install-recommends install \ > + bsdmainutils > + > +echo "dash dash/sh boolean false" | debconf-set-selections > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash > + > +# TODO: check if proxy settings can set outside docker container > +#if [ -n "$http_proxy" ]; then > +# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc > +# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment > +#fi > + > +# ============================================================================== > +# Install Jenkins > +# ============================================================================== > + > +if [ $nojenkins -eq 0 ]; then > + #JENKINS_VERSION=2.164.2 > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab > + JENKINS_VERSION=2.249.3 > + JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2 > + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb > + export JENKINS_UC=https://updates.jenkins.io > + REF=/var/lib/jenkins/plugins > + JENKINS_HOME=/var/lib/jenkins > + JENKINS_PORT=$port > + > + # Jenkins dependencies > + apt-get -q=2 -V --no-install-recommends install \ > + default-jdk daemon psmisc adduser procps unzip > + pip install python-jenkins==1.4.0 > + > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment > + export JENKINS_PORT=$port > + > + groupadd jenkins > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins > + > + wget -nv ${JENKINS_URL} > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - > + > + # allow Jenkins to start and install plugins, as part of dpkg installation > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb > + rm jenkins_${JENKINS_VERSION}_all.deb > + > + source /etc/default/jenkins && \ > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \ > + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins > + > + source /etc/default/jenkins && \ > + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false - > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \ > + if [ -n "$http_proxy" ]; then \ > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \ > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \ > + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" - > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \ > + fi && \ > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins; > + > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins > + > + # set up Jenkins plugins > + cp frontend-install/install-plugins.sh \ > + frontend-install/jenkins-support \ > + frontend-install/clitest \ > + /usr/local/bin/ > + > + mkdir -p $JENKINS_HOME/userContent/docs > + cp frontend-install/config.xml $JENKINS_HOME/config.xml > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml > + chown -R jenkins:jenkins $JENKINS_HOME/ > + > + # start and stop jenkins to pre-populate some settings > + service jenkins start && \ > + sleep 30 && \ > + service jenkins stop > + > + # install other plugins from Jenkins update center > + # NOTE: not sure all of these are needed, but keep list > + # compatible with 1.2.1 release for now > + #/usr/local/bin/install-plugins.sh \ > + # ant:1.9 \ > + # antisamy-markup-formatter:1.5 \ > + # bouncycastle-api:2.17 \ > + # command-launcher:1.3 \ > + # description-setter:1.10 \ > + # display-url-api:2.3.1 \ > + # external-monitor-job:1.7 \ > + # greenballs:1.15 \ > + # icon-shim:2.0.3 \ > + # javadoc:1.5 \ > + # jdk-tool:1.2 \ > + # junit:1.27 \ > + # ldap:1.20 \ > + # mailer:1.23 \ > + # matrix-auth:2.3 \ > + # matrix-project:1.14 \ > + # pam-auth:1.4 \ > + # pegdown-formatter:1.3 \ > + # structs:1.17 \ > + # windows-slaves:1.4 > + > + # install plugins for latest LTS Jenkins (currently 2.249.3) > + /usr/local/bin/install-plugins.sh \ > + script-security \ > + structs \ > + workflow-step-api \ > + workflow-api \ > + junit \ > + scm-api \ > + ant \ > + antisamy-markup-formatter \ > + bouncycastle-api \ > + command-launcher \ > + description-setter \ > + display-url-api \ > + external-monitor-job \ > + greenballs \ > + icon-shim \ > + javadoc \ > + jdk-tool \ > + mailer \ > + matrix-auth \ > + matrix-project \ > + pam-auth \ > + pegdown-formatter \ > + windows-slaves > + > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi > + > + # make the mod.js symlink well after flot is installed > + service jenkins start && sleep 30 && \ > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \ > + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js > + > + chown -R jenkins:jenkins $JENKINS_HOME/ > +fi > + > +# ============================================================================== > +# get ttc script and helpers > +# ============================================================================== > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc > +/usr/local/src/ttc/install.sh /usr/local/bin > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc > + > +# ============================================================================== > +# Serial Config > +# ============================================================================== > +if [ $nojenkins -eq 0 ]; then > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; > cp /usr/local/src/serlogin/serlogin /usr/local/bin' > +else > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln - > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh' > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin > /usr/local/bin' > +fi > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \ > + frontend-install/0002-Output-data-from-port-during-command-execution.patch \ > + /tmp/ > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \ > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;' > + > + > +# ============================================================================== > +# get ebf script and helpers > +# ============================================================================== > +apt-get -q=2 -V --no-install-recommends install jq > +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf > +chmod a+x /usr/local/bin/ebf > + > +# ============================================================================== > +# fserver > +# ============================================================================== > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver > +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server > + > +# ============================================================================== > +# ftc post installation > +# ============================================================================== > +ln -s /fuego-core/scripts/ftc /usr/local/bin/ > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc > +echo ". /etc/bash_completion" >> /root/.bashrc > + > +# ============================================================================== > +# Lava > +# ============================================================================== > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin > diff --git a/install.sh b/install.sh > index a5a90b0..259c887 100755 > --- a/install.sh > +++ b/install.sh > @@ -1,13 +1,13 @@ > #!/bin/bash > # > -# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] > +# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>] > # > > # $1 is the exit code after usage is shown > usage() { > exit_code="$1" > cat <<HERE > -Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] > +Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>] > > Create the docker image and container with the Fuego test distribution. > If no <image_name> is provided, the image will be named 'fuego'. > @@ -34,7 +34,7 @@ fi > > priv=0 > NOCACHE="" > -dockerfile="Dockerfile" > +NOJENKINS="" > > POSITIONAL=() > while [[ $# -gt 0 ]] ; do > @@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do > shift > ;; > --nojenkins) > - dockerfile="Dockerfile.nojenkins" > + NOJENKINS="--nojenkins" > shift > ;; > *) > @@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional arguments > > image_name=${1:-fuego} > jenkins_port=${2:-8090} > +distribution=${3:-debian} > +suite=${4:-stretch} > > container_name="${image_name}-container" > > @@ -79,7 +81,7 @@ fi > > set -e > > -source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile} > +source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS} > if [ "$priv" == "0" ]; then > fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name} > else > diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh > new file mode 100644 > index 0000000..b899e7c > --- /dev/null > +++ b/uninstall-fuego.sh > @@ -0,0 +1,47 @@ > +#!/bin/bash > + > +# Remove Jenkins plugins > +[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins > + > +# uninstall Jenkins > +service jenkins stop > +dpkg -P jenkins > + > +[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh > +[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support > +[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest > +[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins > +sed -i '/^JENKINS_PORT=/d' /etc/environment > + > +# uninstall ttc script > +[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc > + > +# uninstall ebf script > +[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api > +[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf > + > +# uninstall serial config > +[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio > +[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio > +[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp > +[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh > +[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin > +[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin > + > +# uninstall fserver > +[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver > +[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server > + > +# uninstall ftc > +[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc > +[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc > + > +# remove lava target files > +[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup > +[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown > + > +# remove links > +[ -d /fuego-ro ] && rm /fuego-ro > +[ -d /fuego-rw ] && rm /fuego-rw > +[ -d /fuego-core ] && rm /fuego-core > + > -- > 2.20.1 > > The information contained in this e-mail message and in any > attachments/annexure/appendices is confidential to the > recipient and may contain privileged information. > If you are not the intended recipient, please notify the > sender and delete the message along with any > attachments/annexure/appendices. You should not disclose, > copy or otherwise use the information contained in the > message or any annexure. Any views expressed in this e-mail > are those of the individual sender except where the sender > specifically states them to be the views of > Toshiba Software India Pvt. Ltd. (TSIP),Bangalore. > > Although this transmission and any attachments are believed to be > free of any virus or other defect that might affect any computer > system into which it is received and opened, it is the responsibility > of the recipient to ensure that it is virus free and no responsibility > is accepted by Toshiba Embedded Software India Pvt. Ltd, for any loss or > damage arising in any way from its use.