Re: Possibility to log ignored tests

Rules can fill _some_ of the needs of a reporting facility, but
unfortunately, not all: for example, some custom runners ignore Rules
entirely.

Unfortunately, the task of hooking in a custom RunListener can vary
depending on how the tests are being run, and is often not really
possible.  Can you share a little more about your goals?

   David Saff

On Sun, Mar 8, 2015 at 7:06 PM, Nikita Merinov [email protected]
[junit] <[email protected]> wrote:

>
>
> Hi everyone,
>
> I'm trying to write custom reporting facility using JUnit rules.
>
> Quick example:
>
> public class Example {
> @ClassRule
> public static TestWatcher classWatcher = new TestWatcher() {
> @Override
> protected void starting(Description description) {
> // Initialize stuff
> // Log header
> }
>
> @Override
> protected void finished(Description description) {
> // Log footer with metrics from counters (total, succeeded, failed,
> skipped)
> }
>
> };
>
> @Rule
> public TestWatcher testWatcher = new TestWatcher() {
> @Override
> protected void succeeded(Description description) {
> // Log succeeded
> // Increment succeeded counter
> }
>
> @Override
> protected void failed(Throwable e, Description description) {
> // Log failed
> // Increment failed counter
> }
>
> @Override
> protected void skipped(AssumptionViolatedException e, Description
> description) {
> // Log skipped
> // Increment skipped counter
> }
>
> @Override
> protected void starting(Description description) {
> // Log started
> }
>
> @Override
> protected void finished(Description description) {
> // Log failed
> // Increment total counter
> }
> };
>
>     @Test
>     @Ignore
>     public void test1() {}
>
>     @Test
>     public void test2() {}
> }
>
> Issue here is that tests with @Ignore annotation are not handled by JUnit
> rules. Therefore I cannot get any information about ignored tests, and all
> my metrics will be wrong.
>
> From the JUnit source code I can see that if test is marked with @Ignore
> only RunNotifier object will be notified about ignore and that test will
> not be passed to rules.
>
> Got two questions here:
> 1. How can I fix my metrics? The only way I can see is to manually count
> methods with @Ignore annotation via reflection and apply this offset to all
> metrics. Is there any other ways to do that?
> 2. Why presence of @Ignore does not behave like failed assumption? I guess
> that this is because of backward compatibility, is it correct?
>
> Thanks.
>
> --
> Sincerely yours,
> Nikita Merinov
>
>  
>