Re: Slow filenotify-tests

Eli Zaretskii <[email protected]> writes:

Hi Eli,

>> That was discussed years ago, when we have introduced the different
>> SELECT_* variables in test/Makefile. The most common use case for 'make
>> <filename>' is for developers, checking their package. It makes sense to
>> run expensive tests in this case, so I don't like to change it.
>
> So you are saying that "make check" would skip those tests, but
> "make SPECIFIC-tests" would not, is that right?

Yes. This is described in test/README:

--8<---------------cut here---------------start------------->8---
* make check
  Run all tests as defined in the directory.  Expensive and unstable
  tests are suppressed.  The result of the tests for <filename>.el is
  stored in <filename>.log.

* make <filename>  -or-  make <filename>.log
  Run all tests declared in <filename>.el.  This includes expensive
  tests.  In the former case the output is shown on the terminal, in
  the latter case the output is written to <filename>.log.
--8<---------------cut here---------------end--------------->8---

> My problem here is that I rarely run the whole test suite, but very
> frequently run specific tests, especially when some code or some test
> has changed -- to verify it still succeeds.  That's what happened in
> this case as well: I've run filenotify-tests because I've seen your
> changes in them.

That I understand. Therefore my proposal to make it configurable, which
selector is used by default.

>> What we could think about is a configurable preference, which SELECTOR_*
>> variable to apply for 'make <filename>' by default. A 'configure'
>> option?
>
> Is it frequent to run "make <filename>" with the intent to run all of
> the tests in <filename>, even the expensive and unstable ones?

Not the unstable ones, they are not enabled by default with any
$SELECTOR_*. If you want to run *all* tests in filename.el, including
the unstable ones, you shoul use 'SELECTOR=t'.

And yes, this is frequent. While developing something new in a package,
the author profits much from calling 'make <filename>' again and
again. And in etc/CONTRIBUTE, we recommend to run it prior pushing to
the repo:

--8<---------------cut here---------------start------------->8---
It is a good practice to run the unit test of a change prior to committing.
If you have changed, e.g., the file "xt-mouse.el", you can run the unit
tests via

    make && make -C test xt-mouse-tests
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.