Re: bug#48598: 28.0.50; buffer-naming collisions involving bouncers in ERC

"J.P." <[email protected]> writes:

> Hi Michael,

Hi,

> Thanks for patiently explaining yet again. I really should've been more
> mindful of your time and studied up a bit before reaching out. But if
> you'll allow me more excuses, part of what threw me about the
> subdir-discovery situation was that the "normal" stage of the initial
> (new branch) pipeline of fix/bug-48598 didn't include a job named
> test-lisp-erc-erc-d-inotify [1].

These jobs are included via test/infra/test-jobs.yml. And, as the header
line of this file says, it is generated by "make generate-test-jobs".
I'd like to keep this workflow. If you want a new subdirectory erc/erc-d
being handled, you must regenerate this file, and commit it (maybe this
needs to be explained more descriptively?)

As you see in test/infra/Makefile, we apply already special cases for
subdirectories, namely src, eieio, faceup, so-long, and misc. If we need
to handle other special cases, it shall be in that Makefile.

As I see in your patches, you're going this way.

If it is just about your fix/bug-48598 branch, you can change whatever
you want. Of course, you can test the new emba workflow. But if you,
OTOH, just want to run a test for a given bug, you can provide a short
test/infra/gitlab-ci.yml with one or two jobs exactly for this purpose.

> And not that this matters in the slightest, but in an ideal world, *all*
> of ERC's stable tests would *always* run (including the expensive ones),
> both for jobs in diff-based, push pipelines (test-lisp-erc*-inotify) and
> those in the thrice-daily, scheduled ones (test-all-inotify).

It was a design goal to run only non-expensive tests immediately after
applying changes to the sources, in order to see problems fast. That's
why the expensive tests are located in the fat test-all-inotify job
only, running three timws a day. It is the responsibility of the
developer to decide, whether a test is essential, it shouldn't be tagged
as :expensive-test then.

> Also ideal would be having those tests that live in subdirs of
> test/lisp/erc (such as test/lisp/erc/erc-d) run as part of the "main"
> job (test-lisp-erc-inotify) rather than only when some change touches
> their little area.

Again, there are several sub- and subsubdirectories in test/*. It cannot
be decided by the generator how they depend on each other. One case
waiting for optimization is test/lisp/cedet with several
subsubdirectories.

If you are able to define such dependencies in the Makefile, and specify
for erc

    make_params: "-k -C test check-lisp-erc check-lisp-erc-erc-d"

as special case, we're done. And if you really insist in running also
expensive tests, apply

    make_params: "-k -C test check-lisp-erc check-lisp-erc-erc-d SELECTOR='(not (tag :unstable))'"

If you haven't unstable tests, "SELECTOR=t" would work as well.

There is no need to declare additional Makefile targets like
check-expensive-lisp-erc, we have selectors. See for example
test-native-comp-speed0 in test/infra/gitlab-ci.yml.

> FWIW, I've attached some shoddy infra-related patches, mainly as a means
> of better illustrating the aforementioned pie-in-the-sky behavior [2].
> Regardless, I realize that giving ERC special treatment is likely not in
> the cards. As such, I'm planning on rigging up our own CI setup for
> testing proposed changes that hit the bug tracker (especially against
> older Emacs versions) [3]. When the time comes, any guidance you might
> spare will be greatly appreciated.
>
> Thanks,
> J.P.
>
> P.S. I'll try and refrain from bothering you again in the (immediate)
> future.

Thanks for this. These days, I'm affected with some health problems, so
I cannot guarantee any reply in time. That's why I also gave your
patches a cursory reading only, w/o being able to comment on them in
detail just now. But I'll try to reply whenever you have further
proposals; I'm interested in your proposed changes.

> [1] https://emba.gnu.org/emacs/emacs/-/pipelines/16954
>
>     I suppose that's because it was based on a preexisting
>     test/infra/test-jobs.yml (?).

No, you must regenerate and push test/infra/test-jobs.yml as said above.

> [2] That said, a flimsy rationale for the first one might be that it
>     makes it slightly easier on external tooling trying to leverage
>     existing in-tree recipes (but that's probably a stretch). Right now,
>     I'm doing stuff like
>
>     make -C test SELECTOR="(...)" check-lisp-foo
>
>     everywhere. Not a major hassle, but it'd be nice to skip the
>     SELECTOR part, especially when invoking Make by hand. (Just a
>     thought.)

At least for emba recipes, there's no hassle. And also for manual calls
I prefer the SELECTOR approach. In my tramp-tests.el, I have declared
more tags but the "official" ones described in README, so I always need
to discriminmate by SELECTOR. Not a big deal with shell's history.

> [3] If anyone out there cares, it'll also deploy ERC packages built from
>     open bug sets to our own little ELPA to make it easier on everyday
>     folks wanting to give feedback on proposed changes. Actually, we've
>     already been doing all of this for over a year, only this time
>     around, the idea is to make it less amateurish and have it run on
>     Savannah or somewhere other than big cloud infra.

This I don't understand. Perhaps, we can discuss this later (I have a
vague idea on running CI/CD tests for ELPA packages on emba).

Best regards, Michael.