Re: Possibility to log ignored tests

I have a requirement to build custom report (it can be simple text file,
xml, html format, etc) per each execution of test script despite external
runner (i.e. report should be created for all the following: if script was
executed via IDE, via Maven, via remote runner on CI server, etc). This
report will be analyzed further by others.

So, as I understood the only way to provide some listening facility to
external source is to build own custom runner?

On Mon, Mar 9, 2015 at 10:36 PM, David Saff [email protected] [junit] <
[email protected]> wrote:

>
>
> Rules can fill _some_ of the needs of a reporting facility, but
> unfortunately, not all: for example, some custom runners ignore Rules
> entirely.
>
> Unfortunately, the task of hooking in a custom RunListener can vary
> depending on how the tests are being run, and is often not really
> possible.  Can you share a little more about your goals?
>
>    David Saff
>
> On Sun, Mar 8, 2015 at 7:06 PM, Nikita Merinov [email protected]
> [junit] <[email protected]> wrote:
>
>>
>>
>> Hi everyone,
>>
>> I'm trying to write custom reporting facility using JUnit rules.
>>
>> Quick example:
>>
>> public class Example {
>> @ClassRule
>> public static TestWatcher classWatcher = new TestWatcher() {
>> @Override
>> protected void starting(Description description) {
>> // Initialize stuff
>> // Log header
>> }
>>
>> @Override
>> protected void finished(Description description) {
>> // Log footer with metrics from counters (total, succeeded, failed,
>> skipped)
>> }
>>
>> };
>>
>> @Rule
>> public TestWatcher testWatcher = new TestWatcher() {
>> @Override
>> protected void succeeded(Description description) {
>> // Log succeeded
>> // Increment succeeded counter
>> }
>>
>> @Override
>> protected void failed(Throwable e, Description description) {
>> // Log failed
>> // Increment failed counter
>> }
>>
>> @Override
>> protected void skipped(AssumptionViolatedException e, Description
>> description) {
>> // Log skipped
>> // Increment skipped counter
>> }
>>
>> @Override
>> protected void starting(Description description) {
>> // Log started
>> }
>>
>> @Override
>> protected void finished(Description description) {
>> // Log failed
>> // Increment total counter
>> }
>> };
>>
>>     @Test
>>     @Ignore
>>     public void test1() {}
>>
>>     @Test
>>     public void test2() {}
>> }
>>
>> Issue here is that tests with @Ignore annotation are not handled by JUnit
>> rules. Therefore I cannot get any information about ignored tests, and all
>> my metrics will be wrong.
>>
>> From the JUnit source code I can see that if test is marked with @Ignore
>> only RunNotifier object will be notified about ignore and that test will
>> not be passed to rules.
>>
>> Got two questions here:
>> 1. How can I fix my metrics? The only way I can see is to manually count
>> methods with @Ignore annotation via reflection and apply this offset to all
>> metrics. Is there any other ways to do that?
>> 2. Why presence of @Ignore does not behave like failed assumption? I
>> guess that this is because of backward compatibility, is it correct?
>>
>> Thanks.
>>
>> --
>> Sincerely yours,
>> Nikita Merinov
>>
>>
>  
>



-- 
Sincerely yours,
Nikita Merinov