Re: [yocto-patches] [wic][PATCH v3 06/10] tests/docs: add the suite overview README

Paul Barker <[email protected]> Mon, 06 Jul 2026 09:47:56 +0100
Newsgroups	org.yoctoproject.lists.yocto-patches
Message-ID	<[email protected]>
On Wed, 2026-07-01 at 03:40 -0400, Trevor Woerner via
lists.yoctoproject.org wrote:
> The test suite needs a front door: a short document that says what the
> suite is, how it is laid out, and how to run it. This commit adds
> tests/docs/README.md to be that overview.
> 
> The README covers:
> 
>   - the layout of tests/, file by file;
>   - that the suite is unit-only and needs no host tools or build
>     environment, so it runs from a plain checkout;
>   - how to install the test extras and invoke run-tests.sh, pointing at
>     run-tests.sh --help as the authoritative, always-current list of
>     options rather than duplicating them here where they would drift;
>   - that anything run-tests.sh does not recognise is passed through to
>     pytest;
>   - a pointer to linting.md for how ruff is applied;
>   - a small table of the other documents under tests/docs/.
> 
> It deliberately does not enumerate every run-tests.sh option, since the
> runner is expected to grow more of them; the table and the --help
> output stay the source of truth.
> 
> AI-Generated: codex/claude-opus 4.7 (xhigh)
> Signed-off-by: Trevor Woerner <[email protected]>
> ---
> changes in v3:
> - document the standard pytest scratch-directory mechanisms
>   (TMPDIR and --basetemp) in the suite README, so there is no need
>   for a custom temporary-directory variable.
> changes in v2:
> - v1 submitted the entire test suite as a single commit; v2 breaks
>   the work into a reviewable series, and this patch is one step of it.
> ---
>  tests/docs/README.md | 82 ++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 82 insertions(+)
>  create mode 100644 tests/docs/README.md
> 
> diff --git a/tests/docs/README.md b/tests/docs/README.md
> new file mode 100644
> index 000000000000..ba7bd5e192a3
> --- /dev/null
> +++ b/tests/docs/README.md
> @@ -0,0 +1,82 @@
> +# wic test suite
> +
> +A standalone test suite for the wic source tree. It runs from a plain
> +checkout with nothing but `pytest` -- no bitbake, no OpenEmbedded
> +build, and no target image -- so wic's logic can be exercised and kept
> +stable as the code changes.
> +
> +## Contents
> +
> +- [Layout](#layout)
> +- [Running](#running)
> +- [Linting](#linting)
> +- [Documentation](#documentation)

A table of contents like this will easily fall out of sync with the rest
of the file. Claude likes to generate them, but it doesn't always
remember to update them. I don't think it's needed for such a small
file. If you think it's worth keeping, use something like md-toc
(https://github.com/frnmst/md-toc) and integrate it with pre-commit.

> +
> +## Layout
> +
> +```
> +tests/
> +  conftest.py        session banner describing the run
> +  run-tests.sh       wrapper for running the suite
> +  unit/              unit tests that import wic modules directly
> +  docs/              this documentation
> +```
> +
> +The suite is unit-only: every test under `tests/unit/` imports a wic
> +module in-process and asserts on its behaviour, so none of it needs
> +host tools or a build environment.
> +
> +## Running
> +
> +Install wic with its test extras, then run the suite:
> +
> +```bash
> +pip install -e ".[tests]"
> +tests/run-tests.sh
> +```
> +
> +`run-tests.sh` works from anywhere in the checkout. It can also report
> +branch coverage of the wic source; run `tests/run-tests.sh --help` for
> +the current list of options.
> +
> +Anything `run-tests.sh` does not recognise is handed straight to
> +pytest, so the whole pytest command line is available:
> +
> +```bash
> +tests/run-tests.sh -k filemap -v          # one area, verbose
> +tests/run-tests.sh tests/unit             # a single tier or file
> +```
> +
> +## Scratch files
> +
> +Tests that need scratch space use pytest's `tmp_path` fixture, so there
> +is no wic-specific temporary-directory setting. Scratch directories are
> +created under pytest's base temporary directory and cleaned up according
> +to the retention policy in `pyproject.toml` (a passing test's directory
> +is removed, a failing one is kept).
> +
> +By default pytest roots that base directory at the system temporary
> +directory (usually `/tmp`). If that fills up, or you want the scratch
> +files somewhere else, use the standard pytest mechanisms rather than a
> +custom variable. Both are passed straight through by `run-tests.sh`:
> +
> +```bash
> +TMPDIR=/path/to/scratch tests/run-tests.sh   # honoured by pytest
> +tests/run-tests.sh --basetemp=/path/to/scratch
> +```
> +
> +`--basetemp` puts everything under the given directory and clears it at
> +the start of each run; `TMPDIR` keeps the last few sessions there.

Claude loves to write! You can just say that pytest's tmp_path fixture
is used and link to its docs
(https://docs.pytest.org/en/stable/how-to/tmp_path.html#the-tmp-path-fixture).

> +
> +## Linting
> +
> +The test suite is linted with ruff and is held to a clean bar. See
> +[linting.md](linting.md) for the lint modes and how the source tree is
> +treated.
> +
> +## Documentation
> +
> +| File | Content |
> +|------|---------|
> +| [authoring.md](authoring.md) | How to add a unit test to the suite |
> +| [linting.md](linting.md)     | How ruff is used on the suite |

Tables like this add very little value and fall out-of-sync quickly.

Best regards,

-- 
Paul Barker
signature.asc (application/pgp-signature, 252 B)