Method annotations & testing
[email protected] Thu, 17 May 2018 23:19:01 -0400
| Newsgroups | gmane.comp.audio.supercollider.devel |
|---|---|
| Message-ID | <CAB_zQYseAQ8H31V8-YRB-mWvbVwwrpyF8ONuQ+wDbjRPB+ae+A@mail.gmail.com> |
Hey all, The testing thread got me thinking about how we might want to figure out a way to add an annotation feature to SuperCollider. This would give us an easy way to solve a number of issues with UnitTest. This idea is coming partially from Java annotations ( https://en.wikipedia.org/wiki/Java_annotation) which is a compiler-supported form of metadata that can be applied to various things in the language. Two particular use cases for UnitTest are tests requiring the server, and tests that are expected failure. Currently, the only way to get that kind of metadata is by direct listing, which is prone to decay; making it possible for functions to self-register would be much better. I think a generalized and extensible system for annotations could be very helpful. One very basic way to implement it would be by parsing the source code in question prior to the method. This is slow and not particularly robust, but it is easy to implement and could be sped up by caching. I put together a quick example of this idea at this gist: https://gist.github.com/brianlheim/8dec64df924224913b4f60dbc5c3641d If we had this feature available, then some things we could very easily do are: - find all the test methods that require the server (via the @Server annotation) - mark whether a server test is valid for scsynth or supernova only (rare, but plausible) - query whether a method is expected to fail and count its failure as a success/non-failure - actively skip broken tests rather than commenting them out - mark tests that should not run, for example, in CI, because they are too slow or require extra resources - mark tests that require a particular platform (again, rare but plausible) The @Server one seems particularly useful because we could then, if we wanted, rewrite the runTest method to automatically provide a booted and synced server. This would obviously need to be a different annotation than, for instance "@UsesServer", because some tests may still want to boot it themselves. That's maybe going a bit too far, though. The main benefit is the ability to express this in-source and in-language rather than in some separate format. Since the notation is simple and distinctive, it could later be transformed into a real language feature which is parsed at compile time. This is a very open-ended idea and I'm more than welcome to alternative ways of reaching the same result. I feel this is a problem that needs a solution soon, and this is one way of solving it. It would be useful if not for the language in general than at least for the testing framework. Regards, Brian