Re: [Fuego] [PATCH] Docs: Convert the following tbwiki pages into rst format

See comments inline below.  Sorry this review took so long.
I had a conference last week, then took vacation.

See comments inline below.


> -----Original Message-----
> From: Pooja <[email protected]>
> 
> From: Pooja More <[email protected]>
> 
> Following is a list of pages categorized under "feature reference"
> Board_Snapshot
> Dependencies
> Dynamic_Variables
> Fuego_Developer_Notes
> 
> Pages are cross referenced directly using page titles.
> No anchor labels has been used.
has -> have
> 
> Signed-off-by: Pooja More <[email protected]>
> ---
>  docs/rst_src/Board_Snapshot.rst        |  44 +++
>  docs/rst_src/Dependencies.rst          | 241 +++++++++++++
>  docs/rst_src/Dynamic_Variables.rst     |  51 +++
>  docs/rst_src/Fuego_Developer_Notes.rst | 603 +++++++++++++++++++++++++++++++++
>  4 files changed, 939 insertions(+)
>  create mode 100644 docs/rst_src/Board_Snapshot.rst
>  create mode 100644 docs/rst_src/Dependencies.rst
>  create mode 100644 docs/rst_src/Dynamic_Variables.rst
>  create mode 100644 docs/rst_src/Fuego_Developer_Notes.rst
> 
> diff --git a/docs/rst_src/Board_Snapshot.rst b/docs/rst_src/Board_Snapshot.rst
> new file mode 100644
> index 0000000..be0d2d2
> --- /dev/null
> +++ b/docs/rst_src/Board_Snapshot.rst
> @@ -0,0 +1,44 @@
> +
> +
Are these leading blank lines needed?

Are you reserving these in case you need to add an anchor label?

> +###################
> +Board Snapshot
> +###################
> +
> +Fuego provides a feature to grab a "snapshot" of board status
> +information and save that along with other data associated with the
> +run. The idea is that this status information might be helpful for
> +diagnosing the issue when a problem is encountered during a test
> +(either test failure or an error during test execution).
> +
> +This status information is obtained during the 'snapshot' phase of
> +test execution.  The 'snapshot' phase of execution is turned on by
> +default, but if the phases are manually enumerated, this phase can be
> +omitted.
> +
> +The letter 's' is used to specify the 'snapshot' phase of test
> +execution, with the '-p' option in 'ftc run-test'.
``snapshot`` and ``ftc run-test``

> +
> +The default snapshot operation calls
> +:ref:`ov_rootfs_state <function ov rootfs state>`.
> +
> +The board status data is saved in the file: ``machine_snapshot.txt``
> +in the log directory for a run (that is, under /fuego-rw/logs).
directory should also be literal-quoted.

> +
> +==================================
> +Overriding snapshot operations
> +==================================
> +
> +There are two ways to override the snapshot operation, one on a
> +per-board basis and one on a per-test basis.
> +
> +To override the operation on a per-board basis, the function
> +ov_rootfs_state can can be overridden.  This is done by creating a
> +custom distribution overlay file, and then using the DISTRIB variable
``DISTRIB``

> +in the board file for a board.
> +
> +To override the operation on a per-test basis, a test can define its
> +own 'test_snapshot' function.  If defined, then this function will be
``test_snapshot``
> +called in place of the default snapshot function in the Fuego core.
> +
> +See :ref:`test_snapshot  <function_test_snapshot>` for information about how
> +to define this function in a ``fuego_test.sh`` script.
> diff --git a/docs/rst_src/Dependencies.rst b/docs/rst_src/Dependencies.rst
> new file mode 100644
> index 0000000..b3f72e8
> --- /dev/null
> +++ b/docs/rst_src/Dependencies.rst
> @@ -0,0 +1,241 @@
> +
> +
> +####################
> +Dependencies
> +####################
> +
> +===============
> +Introduction
> +===============
> +
> +Fuego includes a test dependency system that allows Fuego to determine
> +whether a test can be run on a board or not.  The test dependency
> +system provides an opportunity for Fuego to do an early abort of a
> +test in case required conditions are not met for the test.
> +
> +The dependency system allows short-circuiting of test execution.  That
> +is, these dependencies are checked during a pre_test phase, and if the
> +dependencies are not met, Fuego aborts the test, before the test is
> +built, deployed and executed on the target.
> +
> +The dependency system consists of 2 parts:
> +
> + 1) A set of test variables in the base script that specify needed attributes
> +    of the device under test. These dependencies are expressed as statically
> +    declared "NEED_" variables in ``fuego_test.sh``
> +
> + 2) The ability to define a test function, :ref:`test_pre_check
> +    <function test pre check>`, that is called before the test executes,
> +    which can test for arbitrary conditions
> +
> +In the future, it intended that this feature will allow for
> +automatically detecting what tests are applicable to particular
> +boards.
> +
> +==================
> +NEED variables
> +==================
> +
> +A test can declare variables (called 'NEED' variables) that describe
> +attributes of the device under test, in the fuego_test.sh script.
> +
> +The following NEED variables are currently supported:
> +
> + * NEED_MEMORY
> + * NEED_FREE_STORAGE
> + * NEED_ROOT
> + * NEED_KCONFIG
> + * NEED_MODULE
> +
> +These variables are usually declared after the source reference
> +definition, and before the first function declaration in ``fuego_test.sh``
> +
> +Declaration example
> +=======================
> +
> +Here is an example, from Benchmark.signaltest:
> +
> +This shows the first few lines of ``fuego_test.sh`` for this test ::
> +
> +  tarball=signaltest.tar.gz
> +
> +  NEED_ROOT=1
> +
> +  function test_build {
> +    make CC="$CC" LD="$LD" LDFLAGS="$LDFLAGS" CFLAGS="$CFLAGS"
> +  }
> +  ...
> +
> +
> +NEED_MEMORY
> +============
> +
> +The NEED_MEMORY variable is used to require that the board have a
``NEED_MEMORY``

> +certain amount of free memory, for the test to run.  The value is
> +expressed in either bytes, kilobytes, megabytes, gigabytes or
> +terabytes.
> +
> +The value is declared as an integer number (base 10) followed by an
> +optional prefix - one of 'K', 'M', 'G', 'T'.
> +
> +Here are some examples:
> +
> + * NEED_MEMORY=500K
> + * NEED_MEMORY=2G
> + * NEED_MEMORY=1500000
> +
> +As a technical detail, the value specified is compared with the value
> +of MemFree in /proc/meminfo of target board.
> +
> +NEED_FREE_STORAGE
> +======================
> +
> +The NEED_FREE_STORAGE variable is used to require that the board have

``NEED_FREE_STORAGE`` (here and below)

> +a certain amount of free storage, in the filesystem where the test
> +needs it, in for the test to run.  The value is expressed in either
> +bytes, kilobytes, megabytes, gigabytes or terabytes.  The value of
> +NEED_FREE_STORAGE is usually provided with 2 strings - a string
> +indicating the required size, and a directory to check.
> +
> +Most tests are executed in the directory specified by $BOARD_TESTDIR,
> +so that is often the second string provided.  However, if a test needs
> +space somewhere else in the filesystem (besides where the test
> +normally runs), this can be specified directly (statically), or via
> +some other test variable.  If no second string is provided, the
> +specified value of free storage required is compared with the amount
> +of available storage in the root filesystem.
> +
> +The value required is declared as an integer number (base 10) followed
> +by an optional prefix - one of 'K', 'M', 'G', 'T'.
> +
> +Here are some examples:
> +
> + * NEED_FREE_STORAGE=2G
> + * NEED_FREE_STORAGE="50M $BOARD_TESTDIR"
> + * NEED_FREE_STORAGE="5T /media/raid-array"
> +
> +As a technical detail, the value specified is compared with the value
> +of the 'Available' column returned by 'df' for the indicated
> +directory.
> +
> +NEED_ROOT
> +==========
> +
> +This variable is declared if a test required to be executed with
> +'root' privileges.  In that case, the following should be added to the
> +test script:
> +
> + * NEED_ROOT=1
> +
> +NEED_KCONFIG
> +=============
> +
> +This variable is used to check that one or more kernel configuration
> +options have specified values.
> +
> +The NEED_KCONFIG line can list more than one kernel config option.

``NEED_KCONFIG``

> +Each option is of the form: CONFIG_OPTION={value}.  Currently, the
> +value must be one of: 'y' or 'n'.
> +
> +Here are some examples:
> +
> + * NEED_KCONFIG="CONFIG_PRINTK=y"
> + * NEED_KCONFIG="CONFIG_LOCKDEP_SUPPORT=n"
> + * NEED_KCONFIG="CONFIG_USB=y CONFIG_USB_EHCI_MV_U20=y"
> +
> +Technical detail:  The kernel configuration is searched for in the
> +following locations, on the target, in order:
> +
> + * /proc/config.gz
> + * /boot/config-$(uname -r) and on the host at:
> + * $KERNEL_SRC/.config
> +
> +If NEED_KCONFIG is defined, but if the kernel configuration of the

``NEED_KCONFIG``

> +target board can not be found, then the dependency check fails.
> +
> +.. note::
> +   it is intended that the kernel build system will set the board
> +   variable KERNEL_SRC, for use by the Fuego system (but this is not
> +   implemented yet).

``KERNEL_SRC``

> +
> +NEED_MODULE
> +===============
> +
> +This variable indicates that a test needs a particular module loaded
> +on the system, in order to run.
> +
> +Here is an example:
> +
> + * NEED_MODULE=bitrev
> +
> +.. note::
> +   it's unclear that this is a good way to detect a kernel feature
> +   needed for a test.  Any module that is upstream can also be included in
> +   the kernel statically.  Testing for a driver or feature as a module would
> +   miss that configuration.
> +
> +==================
> +test_pre_check
> +==================
> +
> +A test base script (fuego_test.sh) can provide a function called
> +:ref:`test_pre_check <function test pre check>`  where arbitrary commands
> +can be run, to determine if the device under test (the board) has the
> +required features or hardware for the test.
> +
> +This function, if present, is run during the pre_test phase of text
> +execution.  Thus, if prerequisite conditions are not met, the test can
> +abort early and avoid the additional test phases (build, deploy, run,
> +etc.)
> +
> +The following functions are commonly used in the test_pre_check routine:
> +
> + * :ref:`assert_define <function assert define>`
> + * :ref:`is_on_target <function is on target>`
> + * :ref:`is_on_target_path <function is on target path>`
> + * :ref:`assert_has_program <function assert has program>`
> + * :ref:`is_on_sdk <function is on sdk>`
> + * :ref:`abort_job <function_abort_job>`

All these function links should have underscores instead of spaces, I believe.
(Let me know if that's not right - I changed them  in the file.)

> +
> +For examples of how to use these functions, see the individual
> +documentation pages for the functions (linked above).
> +
> +Addtionally, the test_pre_check function can execute any additional
> +code it wants (using the 'cmd' function), in order to query the target
> +during this phase, for required capabilities. Or, it might check
> +conditions on the host, network, or extended test environment, to
> +verify that conditions are ready for the test.
> +
> +This might include checking for things like:
> +
> + * mounted file systems
> + * network connections
> + * required hardware
> + * auxiliary test harness availability and preparation
> +
> +======================
> +Envisioned features
> +======================
> +
> +In the future, the test dependency system may be used to allow a Fuego
> +user to select tests which are appropriate for the hardware or
> +distribution that they have.
> +
> +Fuego does not currently have thousands of tests.  But in a future
> +where there ARE thousands of tests, it will be overwhelming to the
> +test user to select those tests which are of interest for their
> +hardware.  The test dependency system will allow Fuego to
> +automatically compare the features or hardware required for a test,
> +with the features and hardware of a board, and decide if a test is
> +compatible or relevant for that board.
> +
> +When Fuego has a test "server", this can be used as a matching service
> +to select tests for execution on boards that have specific features or
> +hardware.  When Fuego has a test "store", then the dependency system
> +can be use to filter the tests to only those that are relevant to
> +their testing needs.
> +
> +The NEED variables are declarative, rather than imperative (like the
> +test_pre_check function), so that it will be possible to develop an
> +automated system to do this matching (between test and board).
> +
> diff --git a/docs/rst_src/Dynamic_Variables.rst b/docs/rst_src/Dynamic_Variables.rst
> new file mode 100644
> index 0000000..97f741a
> --- /dev/null
> +++ b/docs/rst_src/Dynamic_Variables.rst
> @@ -0,0 +1,51 @@
> +
> +
> +###########################
> +Dynamic Varaiables

Varaiables -> Variables

> +###########################
> +
> +"Dynamic variables" in Fuego are variables that can be passed to a
> +test on the command line, and used to customize the operation of a
> +test, for a particular test run.
> +
> +In general testing nomenclature this is referred to as test
> +parameterization.
> +
> +The purpose of dynamic variables is to support "variant" testing,
> +where a script can loop over a test multiple times, changing the
> +variable to different values.
> +
> +In Fuego, during test execution dynamic variable names are expanded to
> +full variables names that are prefixed with the name of the test.  A
> +dynamic variable overrides a spec variable of the same name.
> +
> +Here is an example of using dynamic variables: ::
> +
> +  $ ftc run-test -b beaglebone -t Benchmark.Dhrystone --dynamic-vars "LOOPS=100000000"
> +
> +
> +This would override the default value for BENCHMARK_DHRYSTONE_LOOPS,
> +setting it to 100000000 (100 million) for this run.  Normally, the
> +default spec for Benchmark.Dhrystones specifies a value of 10000000
> +(10 million) for LOOPS.
> +
> +This feature is intended to be useful for doing 'git bisect's of a
> +bug, passing a different git commit id for each iteration of the test.
> +
> +See :ref:`Test_variables <Test variables>` for more information.
> +
> +Notes
> +==========
> +
> +Note that dynamic vars are added to the runtime ``spec.json`` file, which
> +is saved in the log directory for the run being executed.
> +
> +This spec.json file is copied from the one specified for the run
> +(usually from the test's home directory).
> +
> +If dynamic variables have been defined for a test, then they are
> +listed by name in the run-specific spec.json file, as the value of the
> +variable "dyn_vars".  The reason for this is to allow someone who
> +reviews the test results later to easily see whether a particular test
> +variable had a value that derived from the spec, or from a dynamic
> +variable.  This is important for proper results interpretation.
> diff --git a/docs/rst_src/Fuego_Developer_Notes.rst b/docs/rst_src/Fuego_Developer_Notes.rst
> new file mode 100644
> index 0000000..5c1ee70
> --- /dev/null
> +++ b/docs/rst_src/Fuego_Developer_Notes.rst
> @@ -0,0 +1,603 @@
> +
> +################################
> +Fuego Developer Notes
> +################################
> +
> +
> +This page has some detailed notes about Fuego, Jenkins and how they
> +interact:
> +
> +=============
> +Resources
> +=============
> +
> +Here are some pages in this wiki with developer information:
> +
> + * :ref:`Coding style <Coding style>`
> + * :ref:`Core_interfaces <Core interfaces>`
> + * :ref:`Glossary`
> + * :ref:`Fuego test results determination`
> + * :ref:`Fuego_naming_rules <Fuego naming rules>`
> + * :ref:`Fuego Object Details`
> + * :ref:`Integration with ttc`
> + * :ref:`Jenkins User Interface`
> + * :ref:`Jenkins Plugins`
> + * :ref:`License And Contribution Policy`
> + * :ref:`Log files`
> + * :ref:`Metrics`
> + * :ref:`Overlay Generation`
> + * :ref:`ovgen feature notes`
> + * :ref:`Parser module API`
> + * :ref:`Test Script APIs`
> + * :ref:`Test package system`
> + * :ref:`Test server system`
> + * :ref:`Transport notes`
> + * :ref:`Variables`
> +
> +
> +==========
> +Notes
> +==========
> +
> +specific questions to answer
> +===============================
> +
> +What happens when you click on the "run test" button:
> +
> + * what processes start on the host
> +
> +    * java - jar ``/home/jenkins/slave.jar``, executing a shell running
> +      the contents of the job.xml "hudson.tasks.Shell/command" block:
> +
> +      * this block is labeled: "Execute shell: Command" in the "Build"
> +        section of the job, in the configure page for the job in the
> +        Jenkins user interface.
> +
> + * what interface is used between ``prolog.sh`` and the jenkins processes
> +
> +   * stop appears to be by issuing "http://localhost:8090/...stop"
> +   * see :ref:`Fuego-Jenkins`
> +
> +Each Jenkins node is defined in Jenkins
> +in:``/var/lib/jenkins/nodes/config.xml``
> +
> + * The name of the node is used as the "Device" and "NODE_NAME" for a
> +   test.
> +
> +   * These environment variables are passed to the test agent, which is
> +     always "java -jar /home/jenkins/slave.jar"
> +
> + * Who calls ovgen.py - it is included indirectly, when the base
> +   script sources the shell script for it's test type (``functional.sh``
> +   or ``benchmark.sh``)
> +
> +  * base_script sources: ``functional.sh``
> +
> +     * functional.sh sources: ``overlays.sh``
> +
> +       * overlays.sh calls: ``ovgen.py``
> +
> +Jenkins calls:
> +
> +  * java -jar /fuego-core/engine/slave.jar
> +
> +     * with variables:
> +
> +       * Device
> +       * Reboot
> +       * Rebuild
> +       * Target_PreCleanup
> +       * Target_PostCleanup
> +       * TESTDIR
> +       * TESTNAME
> +       * TESTSPEC
> +       * FUEGO_DEBUG
> +
> +   * the Test Run section of the job for a test configuration has a
> +     command with the following shell commands: ::
> +
> +       export Reboot=false
> +       export Rebuild=true
> +       export Target_PreCleanup=true
> +       export Target_PostCleanup=true
> +       export TESTDIR=Functional.bc
> +       export TESTNAME=bc
> +       export TESTSPEC=default
> +       #export FUEGO_DEBUG=1
> +       timeout --signal=9 100m /bin/bash $FUEGO_CORE/engine/tests/${TESTDIR}/${TESTNAME}.sh

Well, this needed updating.  I did that.

> +
> +Some Jenkins notes:Jenkins stores its configuration in plain files
> +under JENKINS_HOME You can edit the data in these files using the web
> +interface, or from the command line using manual editing (and have the
> +changes take affect at runtime by selecting "Reload configuration from
> +disk".
> +
> +By default, Jenkins assumes you are doing a continuous integration
> +action of "build the product, then test the product".   It has default
> +support for Java projects.
> +
> +Fuego seems to use distributed builds (configured in a master/slave
> +fashion).
> +
> +Jenkins home has (from 2007 docs):
> +
> +  * config.xml - has stuff for the main user interface
> +  * *.xml
> +  * fingerprints - directory for artifact fingerprints
> +  * jobs
> +
> +    * <JOBNAME>
> +
> +      * config.xml
> +      * workspace
> +      * latest
> +      * builds
> +
> +        * <ID>
> +        * build.xml
> +        * log
> +        * changelog.xml
> +
> +The docker container interfaces to the outside host filesystem via the
> +following links:
> +
> + * /fuego-ro -> <host-fuego-location>/fuego-ro
> + * /fuego-rw -> <host-fuego-location>/fuego-rw
> + * /fuego-core -> <host-fuego-core-location>
> +
> +What are all the fields in the "configure node" dialog:
> +Specifically:
> +
> + * where is "Description" used?
> + * what is "# of executors"?
> + * how is "Remote FS root" used?
> +
> +   * is this a path inside the Fuego container, or on the target?
> +
> +     * I presume that the slave program is actually 'xxx_prolog.sh', which runs
> +       on host, and that ``/tmp/dev-slave1`` would be where builds for the target
> +       would occur.
> +
> + * what are Labels used for?
> +   as tags for grouping builds
> + * Launch method: Fuego uses the Jenkins option "Launch slave via
> +   execution of command on the Master"
> +   The command is "java -jar /fuego-core/engine/slave.jar"
> +
> +     * NOTE: slave.jar comes from jta-core git repository, under engine/slave.jar
> +
> +
> +The fuego-core repository has: ::
> +
> +
> + engine
> +   overlays - has the base classes for fuego functions
> +     base - has core shell functions
> +     testplans - has json files for parameter specifications
> +     distribs - has shell functions related to the distro
> +   scripts - has fuego scripts and programs
> +    (things like overlays.sh, loggen.py, parser/common.py, ovgen.py, etc.
> +   slave.jar - java program that Jenkins calls to execute a test
> +   tests - has a directory for each test
> +     Benchmark.foo
> +       Benchmark.foo.spec
> +       foo.sh
> +       test.yaml
> +       reference.log
> +       parser.py
> +     Functional.bar
> +     LTP
> +     etc.
> +
> +
> +What is groovy:
> +
> + * an interpreted language for Java, used by the scriptler plugin to
> +   extend Jenkins
> +
> +What plugins are installed with Jenkins in the JTA configuration?
> +
> + * Jenkins Mailer, LDPA, External Monitor Job Type, PAM, Ant, Javadoc
> + * Jenkins Environment File (special)
> + * Credentials, SSH Credentials, Jenkins SSH Slags, SSH Agent
> + * Git Client, Subversion, Token Macro, Maven Integration, CVS
> + * Parameterized Trigger (special)
> + * Git, Groovy Label Assignment Extended Choie Parameter
> + * Rebuilder...
> + * Groovy Postbuild, ez-templates, HTML Publisher (special)
> + * JTA Benchmark show plot plugin (special)
> + * Log Parser Plugin (special)
> + * Dashboard view (special)
> + * Compact Columns (special)
> + * Jenkins Dynamic Parameter (special)
> + * flot (special) - benchmark graphs plotting plug-in for Fuego
> +
> +Which of these did Cogent write?
> +
> + * the flot plugin (not flot itself)
> +
> +What scriptler scripts are included in JTA?
> +
> + * getTargets
> + * getTestplans
> + * getTests
> +
> +What language are scriptler scripts in?
> +
> + * Groovy
> +
> +What is the Maven plugin for Jenkins?
> +
> + * Maven is an apache project to build and manage Java projects
> +
> +   * I don't think the plugin is needed for Fuego
> +
> +Jenkins refers to a "slave" - what does this mean?
> +
> + * it refers to a sub-process that can be delegated work.  Roughly
> +   speaking, Fuego uses the term 'target' instead of 'slave', and
> +   modifies the Jenkins interface to support this.
> +
> +
> +How the tests work
> +===================
> +
> +A simple test that requires no building is Functional.bc
> +
> +  * the test script and test program source are found in the
> +    directory: ``/home/jenkins/tests/Functional.bc``
> +
> +This runs a shell script on target to test the 'bc' program.
> +
> +Functional.bc has the files: ::
> +
> +    bc-script.sh
> +       declares "tarball=bc-script.tar.gz"
> +       defines shell functions:
> +         test_build - calls 'echo' (does nothing)
> +         test_deploy - calls 'put bc-device.sh'
> +         test_run - calls 'assert_define', 'report'
> +           report references bc-device.sh
> +         test_processing - calls 'log_compare'
> +           looking for "OK"
> +       sources $JTA_SCRIPTS_PATH/functional.sh
> +     bc-script.tar.gz
> +       bc-script/bc-device.sh
> +
> +
> +Variables used (in bc-script.sh): ::
> +
> +  BOARD_TESTDIR
> +  TESTDIR
> +  FUNCTIONAL_BC_EXPR
> +  FUNCTIONAL_BC_RESULT
> +
> +
> +
> +A simple test that requires simple building:
> +  Functional.synctest
> +
> +This test tries to call fsync to write data to a file, but is
> +interupted with a kill command during the fsync().  If the child dies
> +before the fsync() completes, it is considered success.
> +
> +It requires shared memory (shmget, shmat) and semaphore IPC (semget
> +and semctl) support in the kernel.
> +
> +Functional synctest has the files: ::
> +
> +     synctest.sh
> +       declares "tarball=synctest.tar.gz"
> +       defines shell functions:
> +         test_build - calls 'make'
> +         test_deploy - calls 'put'
> +         test_run - calls 'assert_define', hd_test_mount_prepare, and 'report'
> +         test_processing - calls 'log_compare'
> +           looking for "PASS : sync interrupted"
> +       sources $JTA_SCRIPTS_PATH/functional.sh
> +     synctest.tar.gz
> +       synctest/synctest.c
> +       synctest/Makefile
> +     synctest_p.log
> +       has "PASS : sync interrupted"
> +
> +
> +Variables used (by synctest.sh) ::
> +
> +  CFLAGS
> +  LDFLAGS
> +  CC
> +  LD
> +  BOARD_TESTDIR
> +  TESTDIR
> +  FUNCTIONAL_SYNCTEST_MOUNT_BLOCKDEV
> +  FUNCTIONAL_SYNCTEST_MOUNT_POINT
> +  FUNCTIONAL_SYNCTEST_LEN
> +  FUNCTIONAL_SYNCTEST_LOOP
> +
> +.. note::
> +
> +  could be improved by checking for CONFIG_SYSVIPC in /proc/config.gz
> +  to verify that the required kernel features are present
> +
> +MOUNT_BLOCKDEV and MOUNT_POINT are used by 'hd_test_mount_prepare' but
> +are prefaced with FUNCTIONAL_SYNCTEST or BENCHMARK_BONNIE
> +
> +
> +from clicking "Run Test", to executing code on the target...
> +config.xml has the slave command: /home/jenkins/slave.jar
> +-> which is a link to /home/jenkins/jta/engine/slave.jar
> +
> +overlays.sh has "run_python $OF_OVGEN ..."
> +where OF_OVGEN is set to "$JTA_SCRIPTS_PATH/ovgen/ovgen.py"
> +
> +How is overlays.sh called?
> +  it is sourced by ``/home/jenkins/scripts/benchmarks.sh`` and
> +    ``/home/jenkins/scripts/functional.sh``
> +
> +``functional.sh`` is sourced by each Funcational.foo script.
> +
> +
> +For Functional.synctest: ::
> +
> +  Functional.synctest/config.xml
> +    for the attribute <hudson.tasks.Shell> (in <builders>)
> +      <command>....
> +        souce $JTA_TESTS_PATH/$JOB_NAME/synctest.sh</command>
> +
> +  synctest.sh
> +    '. $JTA_SCRIPTS_PATH/functional.sh'
> +       'source $JTA_SCRIPTS_PATH/overlays.sh'
> +       'set_overlay_vars'
> +           (in overlays.sh)
> +           run_python $OF_OVGEN ($JTA_SCRIPTS_PATH/ovgen/ovgen.py) ...
> +                  $OF_OUTPUT_FILE ($JTA_SCRIPTS_PATH/work/${NODE_NAME}_prolog.sh)
> +             generate xxx_prolog.sh
> +           SOURCE xxx_prolog.sh
> +
> +       functions.sh pre_test()
> +
> +       functions.sh build()
> +          ... test_build()
> +
> +       functions.sh deploy()
> +
> +       test_run()
> +         assert_define()
> +         functions.sh report()
> +
> +
> +
> +
> +NOTES about ovgen.py
> +======================
> +
> +What does this program do?
> +
> +Here is a sample command line from a test console output: ::
> +
> +  python /home/jenkins/scripts/ovgen/ovgen.py \
> +    --classdir /home/jenkins/overlays//base \
> +    --ovfiles /home/jenkins/overlays//distribs/nologger.dist /home/jenkins/overlays//boards/bbb.board \
> +    --testplan /home/jenkins/overlays//testplans/testplan_default.json \
> +    --specdir /home/jenkins/overlays//test_specs/ \
> +    --output /home/jenkins/work/bbb_prolog.sh
> +
> +
> +So, ovgen.py takes a classdir, a list of ovfiles a testplan and a
> +specdir, and produces a xxx_prolog.sh file, which is then sourced by
> +the main test script
> +
> +Here is information about ovgen.py source: ::
> +
> +  Classes:
> +   OFClass
> +   OFLayer
> +   TestSpecs
> +
> +
> +::
> +
> +  Functions:
> +   parseOFVars - parse Overlay Framework variables and definitions
> +   parseVars - parse variables definitions
> +   parseFunctionBodyName
> +   parseFunction
> +   baseParseFunction
> +   parseBaseFile
> +   parseBaseDir
> +   parseInherit
> +   parseInclude
> +   parseLayerVarOverride
> +   parseLayerFuncOverride
> +   parseLayerVarDefinition
> +   parseLayerCapList - look for BOARD.CAP_LIST
> +   parseOverrideFile
> +   generateProlog
> +   generateSpec
> +   parseGenTestPlan
> +   parseSpec
> +   parseSpecDir
> +   run
> +
> +
> +
> +Sample generated test script
> +==================================
> +
> +bbb_prolog.sh is 195 lines, and has the following vars and functions:
> +::
> +
> +
> +   from class:base-distrib:
> +     ov_get_firmware()
> +     ov_rootfs_kill()
> +     ov_rootfs_drop_caches()
> +     ov_rootfs_oom()
> +     ov_rootfs_sync()
> +     ov_rootfs_reboot()
> +     ov_rootfs_state()
> +     ov_logger()
> +     ov_rootfs_logread()
> +
> +   from class:base-board:
> +    LTP_OPEN_POSIX_SUBTEST_COUNT_POS
> +    MMC_DEV
> +    SRV_IP
> +    SATA_DEV
> +    ...
> +    JTA_HOME
> +    IPADDR
> +    PLATFORM=""
> +    LOGIN
> +    PASSWORD
> +    TRANSPORT
> +    ov_transport_cmd()
> +    ov_transport_put()
> +    ov_transport_get()
> +
> +   from class:base-params:
> +    DEVICE
> +    PATH
> +    SSH
> +    SCP
> +
> +   from class:base-funcs:
> +    default_target_route_setup()
> +
> +   from testplan:default:
> +    BENCHMARK_DHRYSTONE_LOOPS
> +    BENCHMARK_<TESTNAME>_<VARNAME>
> +    ...
> +    FUNCTIONAL_<TESTNAME>_<VARNAME>
> +
> +
> +========
> +Logs
> +========
> +
> +When a test is executed, several different kinds of logs are
> +generated: devlog, systemlogs, the testlogs, and the console log.
> +
> +
> +created by Jenkins
> +====================
> +
> + * console log
> +
> +   * this is located in ``/var/lib/jenkins/jobs/<test_name>/builds/<build_id>/log``
> +   * is has the output from running the test script (on the host)
> +
> +
> +created by ftc
> +=====================
> +
> + * console log
> +
> +   * if 'ftc' was used to run the test, then the console log is
> +     created in the log directory
> +
> +   * it is called consolelog.txt
> +
> +
> +
> +created by the test script
> +================================
> +
> + * these are created in the directory:
> +   ``/fuego-rw/logs/<test_name>/<board>.<spec>.<build_id>.<build_number>/``
> + * devlog has a list of commands run on the board during the test
> +
> +   * named devlog.txt
> +
> + * system logs have the log data from the board (e.g.
> +   /var/log/messages) before and after the test run:
> +
> +   * named: ``syslog.before.txt`` and ``syslog.after.txt``
> +
> + * the test logs have the actual output from the test program on the target
> +
> +   * this is completely dependent on what the test program outputs
> +   * named: ``testlog.txt``
> +
> +     * this is the 'raw' log
> +
> +   * there may be 'parsed' logs, which is the log filtered by log_compare operations:
> +
> +      * this is named: ``testlog.p.txt`` or ``testlog.n.txt``
> +      * the 'p' indicated positive results and the 'n' indicates negative results
> +
> +================
> +Core scripts
> +================
> +
> +The test script is sourced by the Fuego main.sh script
> +
> +This script sources several other scripts, and ends up including
> +``fuego_test.sh`` via the Fuego overlay mechanism
> +
> + * load overlays and set_overlay vars
> + * pre_test $TEST_DIR
> + * build
> + * deploy
> + * test_run
> + * set_testres_file, bench_processing, check_create_logrun (if a benchmark)
> + * get_testlog $TESTDIR, test_processing (if a functional test)
> + * get_testlog $TESTDIR (if a stress test)
> + * test_processing (if a regular test)
> +
> +functions available to test scripts:
> +See :ref:`Test Script APIs`
> +
> +
> +Benchmark tests must provide a parser.py file, which extracts the
> +benchmark results from the log data.
> +
> +It does this by doing the following:
> +import common as plib
> +f = open(plib.TEST_LOG)
> +lines = f.readlines()
> +((parse the data))
> +create a dictionary with a key and value, where the key matches
> +the string in the reference.log file
> +
> +The ``parser.py`` program builds a dictionary of values by parsing
> +the log from the test (basically the test output).
> +It then sends the dictionary, and the pattern for matching the
> +reference log test criteria to the routine:
> +common.py:process_data()
> +
> +It defines ref_section_pat, and passes that to process_data()
> +Here are the different patterns for ref_section_pat: ::
> +
> +  9  "\[[\w]+.[gle]{2}\]"
> +  1  "\[\w*.[gle]{2}\]"
> +  1  "^\[[\d\w_ .]+.[gle]{2}\]"
> +  1  "^\[[\d\w_.-]+.[gle]{2}\]"
> +  1  "^\[[\w\d&._/()]+.[gle]{2}\]"
> +  4  "^\[[\w\d._]+.[gle]{2}\]"
> +  2  "^\[[\w\d\s_.-]+.[gle]{2}\]"
> +  3  "^\[[\w\d_ ./]+.[gle]{2}\]"
> +  5  "^\[[\w\d_ .]+.[gle]{2}\]"
> +  1  "^\[[\w\d_\- .]+.[gle]{2}\]"
> +  1  "^\[[\w]+.[gle]{2}\]"
> +  1  "^\[[\w_ .]+.[gle]{2}\]"
> +
> +
> +Why are so many different ones needed??
> +Why couldn't the syntax be: <var-name> <test> <value> on one line?
> +
> +It turns out this is processed by an 'awk' script.  thus the weird
> +syntax.  We should get rid of the awk script and use python instead.
> +
> +
> +How is benchmarking graphing done?
> +===================================
> +
> +See :ref:`Benchmark parser note`
> +
> +
> +docker tips
> +============
> +
> +See :ref:`Docker Tips`
> --
> 2.7.4
> 
> 
> --

Looks good.  I made the changes I noted above.  Please response to any questions posed.
Looking forward to the next batch.

Thanks,
 -- Tim