Re: [Fuego] Problem with /fuego-rw/tests in your lab

Tim,

We couldn't do a git pull today to get the latest changes as we had some
local changes to get the raspi_kinstall working. Currently, we have moved
to a 64-bit kernel image for rpi board and have made some local changes to
the script in Functional.raspi_kinstall to get it working with a 64-bit
kernel image.

We will do a git pull tomorrow and run our changes on top of it and get
back to you.

Thanks,
Srivatsan


On Mon, Mar 15, 2021 at 8:23 PM <[email protected]> wrote:

> This is good to hear.  Thanks for getting this resolved.
> The errors that Fuego was returning were not very useful
> for determining this problem.  I'd like to think about how
> this condition could be checked for, before or during test
> execution, and reported back with a more useful error message.
>
> > # Version of this test=1.2-4
> It looks like you are still running an old version of fuego_lab_check.
> Can you please 'git pull' the latest master branch of fuego-core?
>
> I plan to do more testing this week.
>
> Thanks.
>  -- Tim
>
>
> > -----Original Message-----
> > From: Pavan Arun Deshpande <[email protected]>
> >
> > Hi Tim,
> >
> > The issue regarding Functional.fuego_lab_check has been resolved.
> > The issue was due to the overlapping build numbers as concerned files
> were already present in the container.
> > Therefore the server was giving an error message saying that " Already
> have a file cannot proceed".
> > Uploading failed because of the above mentioned issue.
> > I was able to resolve this after removing the previous run files.
> >
> > Functional.reboot also had the same issue and this also has been
> resolved.
> >
> > This has been verified with multiple tests and iterations.
> >
> > Please find a copy of log messages generated while running
> Functional.fuego_lab_check
> >
> > ===== doing fuego phase: pre_test =====
> > ===== doing fuego phase: build =====
> > The test is already built
> > Fuego test_build duration=0 seconds
> > ===== doing fuego phase: deploy =====
> > ===== doing fuego phase: snapshot =====
> > Firmware revision: 5.4.83-v8-93349cdff-1+
> > ===== doing fuego phase: run =====
> > -------------------------------------------------
> > TAP version 13
> > 1..1
> > # Version of this test=1.2-4
> > # Location of this test=/fuego-rw/tests/Functional.fuego_lab_check
> > ######### Process list of docker container:
> > USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
> > root         1  0.0  0.0  19424  4544 pts/0    Ss+  11:25   0:00
> /bin/bash
> > jenkins     58  0.0  0.0  18708   184 ?        S    11:25   0:00
> /usr/bin/daemon --name=jenkins --inherit --env=JENKINS_HOME=/var/lib/jenkins
> > --output=/var/log/jenkins/jenkins.log
> --pidfile=/var/run/jenkins/jenkins.pid -- /usr/bin/java
> -Djava.awt.headless=true -
> > Djenkins.install.runSetupWizard=false
> -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true -jar
> > /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war
> --httpPort=8090 --prefix=/fuego
> > jenkins     60  2.4 10.8 9946068 1769860 ?     Sl   11:25   1:31
> /usr/bin/java -Djava.awt.headless=true -
> > Djenkins.install.runSetupWizard=false
> -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true -jar
> > /usr/share/jenkins/jenkins.war --webroot=/var/cache/jenkins/war
> --httpPort=8090 --prefix=/fuego
> > root       110  0.0  0.0   9588   144 ?        Ss   11:25   0:00
> /usr/bin/netserver
> > root       129  0.0  0.0   9440    96 ?        Ss   11:25   0:00 iperf3
> -V -s -D -f M
> > jenkins    176  0.6  0.9 8487928 152384 ?      Sl   11:25   0:23 java
> -jar /fuego-core/slave.jar
> > jenkins    177  0.6  0.9 8421364 150824 ?      Sl   11:25   0:24 java
> -jar /fuego-core/slave.jar
> > jenkins    336  0.0  0.0  11180  2852 ?        S    11:27   0:00 /bin/sh
> -xe /tmp/jenkins184800571287563138.sh
> > jenkins    340  0.0  0.0  11256  3104 ?        S    11:27   0:01
> /bin/bash /fuego-core/scripts/poll_requests.sh -u -d 12h -w 120
> > jenkins   4769  0.6  0.3  99016 56048 ?        S    12:28   0:00
> /usr/bin/python /usr/local/bin/ftc run-request --put-run --allow-upgrade
> > request-2021-03-15_12:27:36.07-pptlab:rpi
> > jenkins   4772  0.3  0.0  12404  4168 ?        S    12:28   0:00
> /bin/bash -e /fuego-core/scripts/main.sh
> > jenkins   4855  0.0  0.0  11192  2848 ?        S    12:28   0:00 bash -c
> { ps -aux; echo $? > /tmp/4772-7340 ; } 2>&1 | tee -a /fuego-
> > rw/logs/Functional.fuego_lab_check/rpi.default.10.10/hostlog.txt
> > jenkins   4856  0.0  0.0  11192   272 ?        S    12:28   0:00 bash -c
> { ps -aux; echo $? > /tmp/4772-7340 ; } 2>&1 | tee -a /fuego-
> > rw/logs/Functional.fuego_lab_check/rpi.default.10.10/hostlog.txt
> > jenkins   4857  0.0  0.0   5832   656 ?        S    12:28   0:00 tee -a
> /fuego-rw/logs/Functional.fuego_lab_check/rpi.default.10.10/hostlog.txt
> > jenkins   4858  0.0  0.0  38304  3316 ?        R    12:28   0:00 ps -aux
> > ok 1 - process list of docker container
> > ######### git describe of fuego-core
> > v1.5.9-9-g79f9df5
> > ok 2 - git describe of fuego-core
> > ######### Installed tests - outside of fuego-core
> > total 4
> > drwxr-xr-x 2 root root 4096 Jan 29 04:40 Functional.fuego_lab_check
> > ok 3 - remotely installed tests
> > ######### Console log of the last run
> > # last run=/fuego-rw/logs/Functional.fuego_board_check/rpi.default.91.91
> > # Log:
> > # ===== doing fuego phase: pre_test =====
> > # ===== doing fuego phase: build =====
> > # The test is already built
> > # Fuego test_build duration=0 seconds
> > # ===== doing fuego phase: deploy =====
> > # ===== doing fuego phase: snapshot =====
> > # Firmware revision: 5.4.83-v8-93349cdff-1+
> > # ===== doing fuego phase: run =====
> > # -------------------------------------------------
> > # == Fuego target feature scan results ==
> > # Information:
> > #   account information: uid=0(root) gid=0(root) groups=0(root)
> > #   working directory: /home/pi/test/fuego.Functional.fuego_board_check
> > #   PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
> > # -------------------------------------
> > # Scan results:
> > # HAS_PROGRAM_CAT=1
> > # HAS_PROGRAM_DF=1
> > # HAS_PROGRAM_FIND=1
> > # HAS_PROGRAM_FREE=1
> > # HAS_PROGRAM_GREP=1
> > # HAS_PROGRAM_HEAD=1
> > # HAS_PROGRAM_LOGGER=1
> > # HAS_PROGRAM_LOGREAD=0
> > # HAS_PROGRAM_MKDIR=1
> > # HAS_PROGRAM_MOUNT=1
> > # HAS_PROGRAM_PS=1
> > # HAS_PROGRAM_RM=1
> > # HAS_PROGRAM_RMDIR=1
> > # HAS_PROGRAM_SYNC=1
> > # HAS_PROGRAM_TAIL=1
> > # HAS_PROGRAM_TEE=1
> > # HAS_PROGRAM_TOUCH=1
> > # HAS_PROGRAM_TRUE=1
> > # HAS_PROGRAM_UMOUNT=1
> > # HAS_PROGRAM_UNAME=1
> > # HAS_PROGRAM_UPTIME=1
> > # HAS_PROGRAM_XARGS=1
> > # HAS_PROGRAM_BRACKET=1
> > # HAS_MKDIR_P=1
> > # HAS_RM_RF=1
> > # HAS_GREP_F=1
> > # HAS_SBIN_REBOOT=1
> > # HAS_BIN_TRUE=1
> > # HAS_SBIN_ROUTE=1
> > # HAS_VAR_LOG_MESSAGES=0
> > # HAS_VAR_LOG_SYSLOG=1
> > # HAS_PROC_INTERRUPTS=1
> > # HAS_PROC_DROP_CACHES=1
> > # HAS_PROC_OOM_SCORE_ADJ=0
> > # HAS_PROC_CONFIG_GZ=0
> > # HAS_MEMORY=935532 K
> > # HAS_STORAGE=15022716 K
> > # Process 1 is program: /sbin/init
> > # -------------------------------------
> > # Based on this scan,
> > # the recommended DISTRIB value should be:
> > #    nosyslogd.dist
> > # -------------------------------------
> > # RESULT=SUCCESS
> > # All required programs, files, features, and permissions are present
> > # -------------------------------------
> > # 10+0 records in
> > # 10+0 records out
> > # 10000 bytes (10 kB, 9.8 KiB) copied, 0.000395734 s, 25.3 MB/s
> > # Data from host:
> > # TRANSPORT=ssh
> > # Time to put 10K file: 0.68
> > # Time to get 10K file: 0.69
> > # Time to do 10 cmds: 6.74
> > # Average time per cmd: 0.67
> > # -------------------------------------------------
> > # ===== doing fuego phase: post_test =====
> > # ===== doing fuego phase: processing =====
> > # log_compare: pattern 'SUCCESS' found 1 times (expected greater or
> equal than 1)
> > # chart config not found. Using default values.
> > # Fuego: requested test phases complete!
> > ######### end of log
> > ok 4 - console log of last test run
> > -------------------------------------------------
> > ===== doing fuego phase: post_test =====
> > scp:
> /home/pi/test/fuego.Functional.fuego_lab_check/Functional.fuego_lab_check.log:
> No such file or directory
> > INFO: the test did not produce a test log on the target
> > ===== doing fuego phase: processing =====
> > chart config not found. Using default values.
> > Fuego: requested test phases complete!
> > Packaging run 'Functional.fuego_lab_check-default-10-rpi'
> > Run packaged successfully, and is at:
> /tmp/run-Functional.fuego_lab_check-default-10-on-pptlab:rpi.frp
> > Run package run-Functional.fuego_lab_check-default-10-on-pptlab:rpi.frp
> was accepted by the server.
> > Run Functional.fuego_lab_check-default-10-rpi put to server
> >
> > Thanks and regards
> >
> > Pavan Arun deshpande
> >
> >
> > On Sat, Mar 13, 2021 at 3:54 AM <[email protected] <mailto:
> [email protected]> > wrote:
> >
> >
> >       There is a problem with running the test:
> Functional.fuego_lab_check
> >       in your lab.  I'll send the error in another message.
> >
> >       Here is the error from the testlog:
> >
> >       ls: cannot access '/fuego-rw/tests': No such file or directory
> >       ----
> >
> >       And here is the report from the console log:
> >
> >       ######### Installed tests (outside of fuego-core)
> >       ls: cannot access '/fuego-rw/tests': No such file or directory
> >       In Fuego signal_handler (due to error in run phase)
> >       ##### doing fuego phase: post_test (from signal handler) #####
> >       ...
> >       -----
> >
> >       This output appears to be from an older version of
> Functional.fuego_lab_check.
> >       One of the things I am testing with this test is the ability to
> update the version
> >       of a test in your lab.  Since there's a problem with the
> /fuego-rw/tests directory,
> >       this operation is not working.
> >
> >       Can you please do 3 things:
> >        1) please check your docker container and make sure that it has
> the directory
> >       /fuego-rw/tests
> >       and that this directory is owned by user jenkins and group jenkins?
> >
> >       2) can you 'git pull' the latest fuego-core from the master
> repository, to pick
> >       up the latest version of the Functional.fuego_lab_check test?
> >
> >       Please make sure that you can run this test locally (from the
> /fuego-core/tests
> >       directory).  That is, just run this test from the updated
> fuego-core repository
> >       in your lab, and confirm that it runs without error.
> >
> >       Once you have done that, I will try some tests to see if I can
> cause your lab
> >       to run upgraded tests.
> >
> >       Thanks,
> >        -- Tim
> >
> >
> >
> >
> >
> > This message contains confidential information and is intended only for
> the individual(s) named. If you are not the intended recipient, you
> > are notified that disclosing, copying, distributing or taking any action
> in reliance on the contents of this mail and attached file/s is strictly
> > prohibited. Please notify the sender immediately and delete this e-mail
> from your system. E-mail transmission cannot be guaranteed to be
> > secured or error-free as information could be intercepted, corrupted,
> lost, destroyed, arrive late or incomplete, or contain viruses. The
> > sender therefore does not accept liability for any errors or omissions
> in the contents of this message, which arise as a result of e-mail
> > transmission.
>
>

-- 






This
message contains confidential information and is intended only 
for the
individual(s) named. If you are not the intended
recipient, you are 
notified that disclosing, copying, distributing or taking any
action in 
reliance on the contents of this mail and attached file/s is strictly
prohibited. Please notify the
sender immediately and delete this e-mail 
from your system. E-mail transmission
cannot be guaranteed to be secured or 
error-free as information could be
intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain
viruses. The sender therefore does 
not accept liability for any errors or
omissions in the contents of this 
message, which arise as a result of e-mail
transmission.