Re: [Fuego] How run Fuego tools on a Jenkins slave node

> -----Original Message-----
> From: Fuego <[email protected]> On Behalf Of Michael Nazzareno Trimarchi
> Sent: Thursday, August 26, 2021 11:38 PM
> To: [email protected]
> Subject: [Fuego] How run Fuego tools on a Jenkins slave node
> 
> Hi
> 
> I have already a ci setup but i would like to use Fuego and run script on slave and get results on master node. Is there a way to configure it?

I believe so.  Other Fuego users have used Fuego with CI systems other than the
default one included (the Jenkins instance inside the Fuego docker container).

You can perform tests using Fuego, using the 'ftc' command line tool.

The default installation of Fuego creates a docker container where Fuego core
scripts and Jenkins are installed.  You can run 'ftc' from inside this container
(which is where the included Jenkins will execute test jobs from), or from outside
the container, from the command line.

When you install Fuego, you have the option to build the container without Jenkins
(so it just consist of the build environment for the tests, and the Fuego core scripts).
See http://fuegotest.org/wiki/Installing_Fuego#without_Jenkins for a description
of the install option used for this.

By way of example, when Fuego is run with Jenkins, it creates Jenkins jobs with
the following command sequence for the "build" attribute (this example is for
the test Functional.hello_world):


#logging areas=pre_test,pre_check,build,makepkg,deploy,snapshot,run,
#    post_test,processing, parser,criteria,charting
#logging levels=debug,verbose,info,warning,error
#export FUEGO_LOGLEVELS="run:debug,parser:verbose"
export FUEGO_CALLER="jenkins"
ftc run-test -b $NODE_NAME -t Functional.hello_world -s default \
    --timeout 6m \
    --reboot false \
    --rebuild false \
    --precleanup true \
    --postcleanup true

If the environment variable FUEGO_CALLER is set to "jenkins", then ftc and the
core test system perform additional steps with the integrated
Jenkins system (such as augmenting the Jenkins build with pointers to Fuego 
test artifacts, so they are accessible in the Jenkins interface).  However, if you
are running Jenkins somewhere else (not in the
Fuego container), then I think the jobs and build data will not be in the
expected directories for Fuego to operate on.  So this aspect of Fuego/Jenkins
integration would be lost (or you would have to recreate it).

If FUEGO_CALLER is not set, then ftc and the core test system will just output
the test execution to stdout, and deposit test artifacts in the fuego-rw/logs
directory.  Build artifacts (e.g. the test program source and build directories
are under fuego-rw/buildzone).

See 'ftc run-test help' for information about the arguments used in the above
example ftc invocation. Not all of these arguments are required (in fact, most
of those will default to the values shown above, but they are placed in the Jenkins
job definition to allow users to more easily remember and edit them for a particular job.)

If I had a BeagleBone Black board in my lab, named 'bbb', I could execute
a test at the command line with: "ftc run-test -b bbb -t hello_world"

By default, visualization of Fuego results (tables and plots) are prepared
for use in the integrated Jenkins interface.  However, you can also generate reports
in various formats, that might be useful with your Jenkins (or whatever other CI
system one might use).

Look at the options for 'ftc gen-report', particularly the 'txt', 'html', or 'csv' output
formats, which might be import-able into your CI system, for generating your
own results data displays.

The graphical results (measure plots) for Benchmark data is generated in a json
format which is interpreted by a javascript module called 'mod.js', which uses
a graphing module called 'flot' that is installed as a Jenkins plugin.  'flot' and 'mod.js'
might be usable with other CI systems that have a web-based user interface
(as they are just javascript (well, jquery)) but it would take some work to perform
that integration.

It sounds like part of your question is about how the host/target relationship is
handled in Fuego. By default, a Fuego test runs on a host, and the test programs
are actually transferred to the device under test and executed there, during test
execution.  However, some Fuego users install the Fuego core system on the actual
device under test, and execute it locally.  In this case, the remote aspect of the
test would be handled by the calling CI system, not by Fuego.

Normally, with the integrated version of Jenkins (the one in the Fuego docker container),
there is a slave used that is assigned to each node (board).  This ends up serializing
the access to that board.  But this slave runs on the host machine, where Jenkins
and Fuego are installed.  Are you asking about executing a Jenkins slave on a remote
machine (perhaps the device under test), and having Fuego installed there?

Hopefully the information above  is helpful.

I'm not sure exactly what you are asking, but please let me know if you need more
information.  I am happy to discuss your configuration, and see if Fuego would work
for you.
 -- Tim