Re: Possibility to log ignored tests

So there probably is a way to get what you want, but it will require a
decent amount of custom code, and some understanding of JUnit's
less-often-used extension points.

Basically, you could create a new ListeningRunner. It would be used like
this:

@RunWith(ListeningRunner.class)
@Listeners(MyListener.class)
public class MyTest {
 // normal test class
}

Where ListeningRunner would say, in part:

class ListeningRunner {
  private final Runner mDelegate;
  private final List<RunListener> mExtraListeners;

  public ListeningRunner(Class testClass) {
    mDelegate = createDelegateRunner(testClass);
    mExtraListeners = createListeners(testClass);
  }

  public abstract void run(RunNotifier notifier) {
    RunNotifier myNotifier = new RunNotifier() {
      public void fireTestStarted(final Description description) throws
StoppedByUserException {
        super.fireTestStarted(description);
        notifier.fireTestStarted(description);
      }

      // and so on for other methods.
    }
    for (RunListener listener : mExtraListeners) {
      myNotifier.addListener(listener);
    }
    mDelegate.run(myNotifier);
  }
}

There's a lot of holes left for you to fill in, unfortunately.

   David Saff


On Tue, Mar 10, 2015 at 12:12 PM, Nikita Merinov [email protected]
[junit] <[email protected]> wrote:

>
>
> I have a requirement to build custom report (it can be simple text file,
> xml, html format, etc) per each execution of test script despite external
> runner (i.e. report should be created for all the following: if script was
> executed via IDE, via Maven, via remote runner on CI server, etc). This
> report will be analyzed further by others.
>
> So, as I understood the only way to provide some listening facility to
> external source is to build own custom runner?
>
> On Mon, Mar 9, 2015 at 10:36 PM, David Saff [email protected] [junit] <
> [email protected]> wrote:
>
>>
>>
>> Rules can fill _some_ of the needs of a reporting facility, but
>> unfortunately, not all: for example, some custom runners ignore Rules
>> entirely.
>>
>> Unfortunately, the task of hooking in a custom RunListener can vary
>> depending on how the tests are being run, and is often not really
>> possible.  Can you share a little more about your goals?
>>
>>    David Saff
>>
>> On Sun, Mar 8, 2015 at 7:06 PM, Nikita Merinov [email protected]
>> [junit] <[email protected]> wrote:
>>
>>>
>>>
>>> Hi everyone,
>>>
>>> I'm trying to write custom reporting facility using JUnit rules.
>>>
>>> Quick example:
>>>
>>> public class Example {
>>> @ClassRule
>>> public static TestWatcher classWatcher = new TestWatcher() {
>>> @Override
>>> protected void starting(Description description) {
>>> // Initialize stuff
>>> // Log header
>>> }
>>>
>>> @Override
>>> protected void finished(Description description) {
>>> // Log footer with metrics from counters (total, succeeded, failed,
>>> skipped)
>>> }
>>>
>>> };
>>>
>>> @Rule
>>> public TestWatcher testWatcher = new TestWatcher() {
>>> @Override
>>> protected void succeeded(Description description) {
>>> // Log succeeded
>>> // Increment succeeded counter
>>> }
>>>
>>> @Override
>>> protected void failed(Throwable e, Description description) {
>>> // Log failed
>>> // Increment failed counter
>>> }
>>>
>>> @Override
>>> protected void skipped(AssumptionViolatedException e, Description
>>> description) {
>>> // Log skipped
>>> // Increment skipped counter
>>> }
>>>
>>> @Override
>>> protected void starting(Description description) {
>>> // Log started
>>> }
>>>
>>> @Override
>>> protected void finished(Description description) {
>>> // Log failed
>>> // Increment total counter
>>> }
>>> };
>>>
>>>     @Test
>>>     @Ignore
>>>     public void test1() {}
>>>
>>>     @Test
>>>     public void test2() {}
>>> }
>>>
>>> Issue here is that tests with @Ignore annotation are not handled by
>>> JUnit rules. Therefore I cannot get any information about ignored tests,
>>> and all my metrics will be wrong.
>>>
>>> From the JUnit source code I can see that if test is marked with @Ignore
>>> only RunNotifier object will be notified about ignore and that test will
>>> not be passed to rules.
>>>
>>> Got two questions here:
>>> 1. How can I fix my metrics? The only way I can see is to manually count
>>> methods with @Ignore annotation via reflection and apply this offset to all
>>> metrics. Is there any other ways to do that?
>>> 2. Why presence of @Ignore does not behave like failed assumption? I
>>> guess that this is because of backward compatibility, is it correct?
>>>
>>> Thanks.
>>>
>>> --
>>> Sincerely yours,
>>> Nikita Merinov
>>>
>>>
>>
>
>
> --
> Sincerely yours,
> Nikita Merinov
>
>  
>