Re: [Fuego] Error in running the test after latest pull

Hi Tim,

Issues got fixed with the suggested changes and with recent pull. We tested
a case by logging in a request in
fserver for a test to be run on rpi. This test also went smooth, but the
results could not be
uploaded to the server due to the fserver issue.

Please find the logs below:

ftc run-request --put-run
Trying to get request 'request-2020-12-10_05:36:12.67-himanshu_wks:rpi'
from server
TRB: best_local_match=('', u'Benchmark.Dhrystone', '1.1-1')
TRB: best_remote_match=None
TRB: candidate=('', u'Benchmark.Dhrystone', '1.1-1')
Executing test Benchmark.Dhrystone on board rpi (using spec default)
Running test 'Benchmark.Dhrystone' on board 'rpi' using spec 'default'
===== doing fuego phase: pre_test =====
===== doing fuego phase: build ========
The test is already built
Fuego test_build duration=0 seconds
===== doing fuego phase: deploy =====
===== doing fuego phase: snapshot =====
Firmware revision: 5.4.81-v7-40+
===== doing fuego phase: run =====
-------------------------------------------------

Dhrystone Benchmark, Version 2.1 (Language: C)

Program compiled without 'register' attribute

Please give the number of runs through the benchmark:
Execution starts, 10000000 runs through Dhrystone
Execution ends

Final values of the variables used in the benchmark:

Int_Glob:            5
        should be:   5
Bool_Glob:           1
        should be:   1
Ch_1_Glob:           A
        should be:   A
Ch_2_Glob:           B
        should be:   B
Arr_1_Glob[8]:       7
        should be:   7
Arr_2_Glob[8][7]:    10000010
        should be:   Number_Of_Runs + 10
Ptr_Glob->
  Ptr_Comp:          10326352
        should be:   (implementation-dependent)
  Discr:             0
        should be:   0
  Enum_Comp:         2
        should be:   2
  Int_Comp:          17
        should be:   17
  Str_Comp:          DHRYSTONE PROGRAM, SOME STRING
        should be:   DHRYSTONE PROGRAM, SOME STRING
Next_Ptr_Glob->
  Ptr_Comp:          10326352
        should be:   (implementation-dependent), same as above
  Discr:             0
        should be:   0
  Enum_Comp:         1
        should be:   1
  Int_Comp:          18
        should be:   18
  Str_Comp:          DHRYSTONE PROGRAM, SOME STRING
        should be:   DHRYSTONE PROGRAM, SOME STRING
Int_1_Loc:           5
        should be:   5
Int_2_Loc:           13
        should be:   13
Int_3_Loc:           7
        should be:   7
Enum_Loc:            1
        should be:   1
Str_1_Loc:           DHRYSTONE PROGRAM, 1'ST STRING
        should be:   DHRYSTONE PROGRAM, 1'ST STRING
Str_2_Loc:           DHRYSTONE PROGRAM, 2'ND STRING
        should be:   DHRYSTONE PROGRAM, 2'ND STRING

Microseconds for one run through Dhrystone:    1.0
Dhrystones per Second:                      1000000.0

-------------------------------------------------
===== doing fuego phase: post_test =====
===== doing fuego phase: processing =====
Going to check 1 fail cases for rpi.default.Benchmark.Dhrystone
No fail cases detected for rpi.default.Benchmark.Dhrystone
Fuego: requested test phases complete!
Packaging run 'Benchmark.Dhrystone-default-16-rpi'
run/
run/testlog.txt
run/machine-snapshot.txt
run/build.xml
run/syslog.after.txt
run/prolog.sh
run/devlog.txt
run/spec.json
run/consolelog.txt
run/run.json
run/syslog.before.txt
Run packaged successfully, and is at:
/tmp/run-Benchmark.Dhrystone-default-16-on-himanshu_wks:rpi.frp
url = http://birdcloud.org/cgi-bin/fserver.py/?action=put_run
file to send= run-Benchmark.Dhrystone-default-16-on-himanshu_wks:rpi.frp
response from server = <Response [500]>
Traceback (most recent call last):
  File "/usr/local/bin/ftc", line 6175, in <module>
    main()
  File "/usr/local/bin/ftc", line 6037, in main
    do_run_request(conf, options)
  File "/usr/local/bin/ftc", line 3553, in do_run_request
    do_put_run(conf, [run_id])
  File "/usr/local/bin/ftc", line 4223, in do_put_run
    result, content = put_run_fuego(conf, run_filepath)
ValueError: need more than 1 value to unpack

Thanks and regards,
Pooja More


On Thu, Dec 10, 2020 at 12:10 AM Bird, Tim <[email protected]> wrote:

>
>
> > -----Original Message-----
> > From: Pooja Sanjay More <[email protected]>
> > Sent: Wednesday, December 9, 2020 12:08 AM
> > To: [email protected]; Bird, Tim <[email protected]>;
> [email protected]
> > Subject: Error in running the test after latest pull
> >
> > Hi Tim,
> >
> > While running a Functional.hello_world test on the rpi board  of pptlab
> we are getting the following errors. We are getting this after the
> > latest changes were pulled around 8-10 days back.
> >
> >
> > Please find the logs below.
> >
> This looks like 4 different issues:
>  1. cannot change locale
>  2. permission denied on build lock
>  3. no test log on target
>  4. file exists error on console log
>
> One or more of these might be "cascade" errors, which means that some other
> error is causing these, and the error is just a symptom of another error.
>
> I'll comment on each of these inline below.
>
> > Logs:
> >
> > root@cpu-366U:/#  ftc run-test -b rpi -t Functional.hello_world
> > Running test 'Functional.hello_world' on board 'rpi' using spec 'default'
> > ===== doing fuego phase: pre_test =====
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
>
> From the frequency of these, this looks like a host-side problem (and not
> a target-side
> problem).  However, I could be wrong.
>
> Can you do the following, and send me the results:
> (inside the container):
>
> ls -l /etc/environment ; cat /etc/environment
> ls -l /etc/locale.gen ; cat /etc/locale.gen
> ls -l /etc/locale.conf ; cat /etc/locale.conf
>
> Mine are basically empty (except for the JENKINS_PORT) in my Docker
> container.
>
> Please try one of the fixes below (inside the Docker container):
> fix 1:
> perl -pi -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/g' /etc/locale.gen
> locale-gen en_US.UTF-8
> update-locale en_US.UTF-8
>
> fix 2:
> Add support for en_US.UTF-8 inside the container.
>
> echo "LC_ALL=en_US.UTF-8" >> /etc/environment
> echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen
> echo "LANG=en_US.UTF-8" > /etc/locale.conf
> locale-gen en_US.UTF-8
>
> This worked for me.

(if you don't have locale-gen in the container, try "apt-get install
> locales")
>
> If this doesn't fix it, try these steps on the raspberry pi board.
>
> Please report back which of these fixes (if any) worked.
>
> If it still doesn't fix it, then please get a full debug log of the run,
> and
> send that.
> $ ftc --debug run-test -b rpi -t hello_world
>
> Note that you don't need to use the full test name
> "Functional.hello_world",
> but only the shortened name.
>
>
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > ===== doing fuego phase: build =====
> > /fuego-core/scripts/functions.sh: line 403:
> /fuego-rw/buildzone/rpi.default.Functional.hello_world-/fuego.build.lock:
> Permission denied
>
> This sounds like it was from a test that failed during the build phase and
> left
> its lock file hanging around.
>
> Try just removing it:
> rm /fuego-rw/buildzone/rpi.default.Functional.hello_world-/fuego.build.lock
>
> Also, there should be a toolchain name in the directory path for the build
> directory.
> Does your rpi.board file specify a TOOLCHAIN setting?
> # cat /fuego-rw/boards/rpi3-2.board | grep TOOL
> TOOLCHAIN="debian-armhf"
>
> (You can also use ftc to get this:
>  # ftc query-board -b rpi3-2 -n TOOLCHAIN
> debian-armhf
>
> It looks like there's a bug in fuego-core/scripts/common.sh where the
> TOOLCHAIN
> is not set before it is used to set the value of JOB_BUILD_DIR.  I made a
> fix and committed it.
>
> I'm not sure what change I made that would have affect this.
>
> > In Fuego signal_handler (due to error in build phase)
> > ##### doing fuego phase: post_test (from signal handler) #####
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > scp:
> /home/pi/test/fuego.Functional.hello_world/Functional.hello_world.log: No
> such file or directory
>
> I'd like to see the full log to diagnose by the test log is not being
> created on the target.
> Please run the test with '--debug' and send the log.
>
> > INFO: the test did not produce a test log on the target
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)
> > ##### doing fuego phase: processing (from signal handler) #####
> > ### WARNING: Program returned exit code ''
> > ### WARNING: Log evaluation may be invalid
> > !!! ERROR: ERROR: log_compare: pattern 'SUCCESS' found 0 times (expected
> greater or equal than 1)
> > ERROR: test_processing returned an error
> > chart config not found. Using default values.
> > ERROR: results did not satisfy the threshold
> > Traceback (most recent call last):
> >   File "/usr/local/bin/ftc", line 6162, in <module>
> >     main()
> >   File "/usr/local/bin/ftc", line 6051, in main
> >     rcode, run_id = do_run_test(conf, options)
> >   File "/usr/local/bin/ftc", line 4710, in do_run_test
> >     os.symlink(log_filename, jenkins_log_filename)
> > OSError: [Errno 17] File exists
>
> This looks like an issue with there being an already existing run for a
> test.
> If so, I need to handle this issue better (or fail more gracefully).
>
> Can you do the following, inside the container:
>
>  $ ls -l /fuego-rw/logs/Functional.hello_world
> and
>  $ ls -l /var/lib/jenkins/jobs/*hello_world*/builds
>
> and send me the results?
>
> Sometimes this issue clears itself up as you execute more jobs
> (from either Jenkins or using ftc on the command line).
>
> However, ftc is supposed to catch this and avoid this error.
>
> I added some code to try to adjust the build number better
> depending on the state of both Jenkins build numbers and
> Fuego log (/run) numbers.
>
> Please do a 'git pull' on the fuego-core  repository, and try again.
>
> Thanks very much for the bug report.
>  -- Tim
>
>

-- 






This
message contains confidential information and is intended only 
for the
individual(s) named. If you are not the intended
recipient, you are 
notified that disclosing, copying, distributing or taking any
action in 
reliance on the contents of this mail and attached file/s is strictly
prohibited. Please notify the
sender immediately and delete this e-mail 
from your system. E-mail transmission
cannot be guaranteed to be secured or 
error-free as information could be
intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain
viruses. The sender therefore does 
not accept liability for any errors or
omissions in the contents of this 
message, which arise as a result of e-mail
transmission.