Re: Possibility to log ignored tests

Understood, thank you.

Sincerely yours,
Nikita Merinov

On Wed, Mar 11, 2015 at 5:25 PM, David Saff [email protected] [junit] <
[email protected]> wrote:

>
>
> So there probably is a way to get what you want, but it will require a
> decent amount of custom code, and some understanding of JUnit's
> less-often-used extension points.
>
> Basically, you could create a new ListeningRunner. It would be used like
> this:
>
> @RunWith(ListeningRunner.class)
> @Listeners(MyListener.class)
> public class MyTest {
>  // normal test class
> }
>
> Where ListeningRunner would say, in part:
>
> class ListeningRunner {
>   private final Runner mDelegate;
>   private final List<RunListener> mExtraListeners;
>
>   public ListeningRunner(Class testClass) {
>     mDelegate = createDelegateRunner(testClass);
>     mExtraListeners = createListeners(testClass);
>   }
>
>   public abstract void run(RunNotifier notifier) {
>     RunNotifier myNotifier = new RunNotifier() {
>       public void fireTestStarted(final Description description) throws
> StoppedByUserException {
>         super.fireTestStarted(description);
>         notifier.fireTestStarted(description);
>       }
>
>       // and so on for other methods.
>     }
>     for (RunListener listener : mExtraListeners) {
>       myNotifier.addListener(listener);
>     }
>     mDelegate.run(myNotifier);
>   }
> }
>
> There's a lot of holes left for you to fill in, unfortunately.
>
>    David Saff
>
>
> On Tue, Mar 10, 2015 at 12:12 PM, Nikita Merinov [email protected]
> [junit] <[email protected]> wrote:
>
>>
>>
>> I have a requirement to build custom report (it can be simple text file,
>> xml, html format, etc) per each execution of test script despite external
>> runner (i.e. report should be created for all the following: if script was
>> executed via IDE, via Maven, via remote runner on CI server, etc). This
>> report will be analyzed further by others.
>>
>> So, as I understood the only way to provide some listening facility to
>> external source is to build own custom runner?
>>
>> On Mon, Mar 9, 2015 at 10:36 PM, David Saff [email protected] [junit] <
>> [email protected]> wrote:
>>
>>>
>>>
>>> Rules can fill _some_ of the needs of a reporting facility, but
>>> unfortunately, not all: for example, some custom runners ignore Rules
>>> entirely.
>>>
>>> Unfortunately, the task of hooking in a custom RunListener can vary
>>> depending on how the tests are being run, and is often not really
>>> possible.  Can you share a little more about your goals?
>>>
>>>    David Saff
>>>
>>> On Sun, Mar 8, 2015 at 7:06 PM, Nikita Merinov [email protected]
>>> [junit] <[email protected]> wrote:
>>>
>>>>
>>>>
>>>> Hi everyone,
>>>>
>>>> I'm trying to write custom reporting facility using JUnit rules.
>>>>
>>>> Quick example:
>>>>
>>>> public class Example {
>>>> @ClassRule
>>>> public static TestWatcher classWatcher = new TestWatcher() {
>>>> @Override
>>>> protected void starting(Description description) {
>>>> // Initialize stuff
>>>> // Log header
>>>> }
>>>>
>>>> @Override
>>>> protected void finished(Description description) {
>>>> // Log footer with metrics from counters (total, succeeded, failed,
>>>> skipped)
>>>> }
>>>>
>>>> };
>>>>
>>>> @Rule
>>>> public TestWatcher testWatcher = new TestWatcher() {
>>>> @Override
>>>> protected void succeeded(Description description) {
>>>> // Log succeeded
>>>> // Increment succeeded counter
>>>> }
>>>>
>>>> @Override
>>>> protected void failed(Throwable e, Description description) {
>>>> // Log failed
>>>> // Increment failed counter
>>>> }
>>>>
>>>> @Override
>>>> protected void skipped(AssumptionViolatedException e, Description
>>>> description) {
>>>> // Log skipped
>>>> // Increment skipped counter
>>>> }
>>>>
>>>> @Override
>>>> protected void starting(Description description) {
>>>> // Log started
>>>> }
>>>>
>>>> @Override
>>>> protected void finished(Description description) {
>>>> // Log failed
>>>> // Increment total counter
>>>> }
>>>> };
>>>>
>>>>     @Test
>>>>     @Ignore
>>>>     public void test1() {}
>>>>
>>>>     @Test
>>>>     public void test2() {}
>>>> }
>>>>
>>>> Issue here is that tests with @Ignore annotation are not handled by
>>>> JUnit rules. Therefore I cannot get any information about ignored tests,
>>>> and all my metrics will be wrong.
>>>>
>>>> From the JUnit source code I can see that if test is marked with
>>>> @Ignore only RunNotifier object will be notified about ignore and that test
>>>> will not be passed to rules.
>>>>
>>>> Got two questions here:
>>>> 1. How can I fix my metrics? The only way I can see is to manually
>>>> count methods with @Ignore annotation via reflection and apply this offset
>>>> to all metrics. Is there any other ways to do that?
>>>> 2. Why presence of @Ignore does not behave like failed assumption? I
>>>> guess that this is because of backward compatibility, is it correct?
>>>>
>>>> Thanks.
>>>>
>>>> --
>>>> Sincerely yours,
>>>> Nikita Merinov
>>>>
>>>>
>>>
>>
>>
>> --
>> Sincerely yours,
>> Nikita Merinov
>>
>>
>  
>