| Newsgroups |
gmane.comp.java.junit.user |
| Message-ID |
<CAM21Rt96yaM44dQa0EvkhQOVw5+=fnr1uUa1gL9pYbgxXbFAjA@mail.gmail.com> |
--001a113f9070ed3b340522ab8c2d
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Hi Andreas,
This is not going to be useful if you're using plain JUnit but you can look
at the solution in randomizedtesting -- we use it in Lucene and Solr and it
works fairly well.
You do need to have a common superclass or repeat the rule annotation on
every class this should be applied to, unfortunately. This is the
annotation:
https://github.com/apache/lucene-solr/blob/trunk/lucene/test-framework/src/=
java/org/apache/lucene/util/LuceneTestCase.java#L206
this annotation is used for suppressing it (for a class):
https://github.com/apache/lucene-solr/blob/trunk/lucene/test-framework/src/=
java/org/apache/lucene/util/LuceneTestCase.java#L352-L365
and this is rule declaration chain:
https://github.com/apache/lucene-solr/blob/trunk/lucene/test-framework/src/=
java/org/apache/lucene/util/LuceneTestCase.java#L581
The scenario in Lucene is quite complex, for example you want to suppress
certain checks on failures (other rules are permitted to write diagnostics
to sysout if something fails), but you can look for the inspiration on how
it was done here:
https://github.com/apache/lucene-solr/blob/trunk/lucene/test-framework/src/=
java/org/apache/lucene/util/TestRuleLimitSysouts.java
If you peek at the source code and the comments you'll see the difficulties
this entails (early-on stolen references to System.out, running with a
security manager, etc.).
Hope this helps somehow,
Dawid
On Wed, Oct 21, 2015 at 2:07 PM, Andreas Worm [email protected]
[junit] <[email protected]> wrote:
>
>
> Hi,
>
> I am looking for a way to fail unittests when they write to System.out
> or System.err. I found different ways to achieve that and tried a
> solution with a RunListener, because these can be configured using the
> maven-surefire-plugin. Now I discovered, that RunListeners can't really
> fail tests because fail() throws an AssertionException and Exceptions
> remove the RunListener. Is there a better way to discover output?
>
> To summarize what I want:
> * a way to discover output via System.out without preventing Logger
> functionality
> * this way should not involve huge code changes on the testclasses (
> like adding TestRules everywhere)
> * all unittests which produced output should be reported as "failing"
> when you run with `mvn test`
>
> I tried to answer these questions on SO but got only a few helpful
> responses, so I thought I ask the specialists here and get more
> opinions. For reference the SO questions:
> * http://stackoverflow.com/questions/33214683
> * http://stackoverflow.com/questions/33207996
>
> Any help on this topic is very appreciated.
>
> Regards
> Andreas Worm
>
>=20
>
--001a113f9070ed3b340522ab8c2d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<head>
<style type=3D"text/css">
<!--
/* start of attachment style */
.ygrp-photo-title{
clear: both;
font-size: smaller;
height: 15px;
overflow: hidden;
text-align: center;
width: 75px;
}
div.ygrp-photo{
background-position: center;
background-repeat: no-repeat;
background-color: white;
border: 1px solid black;
height: 62px;
width: 62px;
}
div.photo-title=20
a,
div.photo-title a:active,
div.photo-title a:hover,
div.photo-title a:visited {
text-decoration: none;=20
}
div.attach-table div.attach-row {
clear: both;
}
div.attach-table div.attach-row div {
float: left;
/* margin: 2px;*/
}
p {
clear: both;
padding: 15px 0 3px 0;
overflow: hidden;
}
div.ygrp-file {
width: 30px;
valign: middle;
}
div.attach-table div.attach-row div div a {
text-decoration: none;
}
div.attach-table div.attach-row div div span {
font-weight: normal;
}
div.ygrp-file-title {
font-weight: bold;
}
/* end of attachment style */
-->
</style>
</head>
<html>
<head>
<style type=3D"text/css">
<!--
#ygrp-mkp {
border: 1px solid #d8d8d8;
font-family: Arial;
margin: 10px 0;
padding: 0 10px;
}
#ygrp-mkp hr {
border: 1px solid #d8d8d8;
}
#ygrp-mkp #hd {
color: #628c2a;
font-size: 85%;
font-weight: 700;
line-height: 122%;
margin: 10px 0;
}
#ygrp-mkp #ads {
margin-bottom: 10px;
}
#ygrp-mkp .ad {
padding: 0 0;
}
#ygrp-mkp .ad p {
margin: 0;
}
#ygrp-mkp .ad a {
color: #0000ff;
text-decoration: none;
}
-->
</style>
</head>
<body>
<!-- |**|begin egp html banner|**| -->
<br><br>
<!-- |**|end egp html banner|**| -->
<div dir=3D"ltr"><br><div>Hi Andreas,</div><div><br></div><div>This is not =
going to be useful if you're using plain JUnit but you can look at the =
solution in randomizedtesting -- we use it in Lucene and Solr and it works =
fairly well.=C2=A0</div><div><br></div><div>You do need to have a common su=
perclass or repeat the rule annotation on every class this should be applie=
d to, unfortunately. This is the annotation:</div><div><br></div><div><a hr=
ef=3D"https://github.com/apache/lucene-solr/blob/trunk/lucene/test-framewor=
k/src/java/org/apache/lucene/util/LuceneTestCase.java#L206">https://github.=
com/apache/lucene-solr/blob/trunk/lucene/test-framework/src/java/org/apache=
/lucene/util/LuceneTestCase.java#L206</a><br></div><div><br></div><div>this=
annotation is used for suppressing it (for a class):</div><div><br></div><=
div><a href=3D"https://github.com/apache/lucene-solr/blob/trunk/lucene/test=
-framework/src/java/org/apache/lucene/util/LuceneTestCase.java#L352-L365">h=
ttps://github.com/apache/lucene-solr/blob/trunk/lucene/test-framework/src/j=
ava/org/apache/lucene/util/LuceneTestCase.java#L352-L365</a><br></div><div>=
<br></div><div>and this is rule declaration chain:</div><div><br></div><div=
><a href=3D"https://github.com/apache/lucene-solr/blob/trunk/lucene/test-fr=
amework/src/java/org/apache/lucene/util/LuceneTestCase.java#L581">https://g=
ithub.com/apache/lucene-solr/blob/trunk/lucene/test-framework/src/java/org/=
apache/lucene/util/LuceneTestCase.java#L581</a><br></div><div><br></div><di=
v>The scenario in Lucene is quite complex, for example you want to suppress=
certain checks on failures (other rules are permitted to write diagnostics=
to sysout if something fails), but you can look for the inspiration on how=
it was done here:</div><div><br></div><div><a href=3D"https://github.com/a=
pache/lucene-solr/blob/trunk/lucene/test-framework/src/java/org/apache/luce=
ne/util/TestRuleLimitSysouts.java">https://github.com/apache/lucene-solr/bl=
ob/trunk/lucene/test-framework/src/java/org/apache/lucene/util/TestRuleLimi=
tSysouts.java</a><br></div><div><br></div><div>If you peek at the source co=
de and the comments you'll see the difficulties this entails (early-on =
stolen references to System.out, running with a security manager, etc.).</d=
iv><div><br></div><div>Hope this helps somehow,</div><div>Dawid</div><div><=
br></div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On=
Wed, Oct 21, 2015 at 2:07 PM, Andreas Worm <a href=3D"mailto:andreas.worm@=
headissue.com">[email protected]</a> [junit] <span dir=3D"ltr"><=
;<a href=3D"mailto:[email protected]" target=3D"_blank">junit@yahoogrou=
ps.com</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"=
margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<u></u>
=20
<div style=3D"background-color:#fff">
<span>=C2=A0</span>
<div>
<div>
<div>
=20=20=20=20=20=20
=20=20=20=20=20=20
<p>Hi,<br>
<br>
I am looking for a way to fail unittests when they write to System.out <br>
or System.err. I found different ways to achieve that and tried a <br>
solution with a RunListener, because these can be configured using the <br>
maven-surefire-plugin. Now I discovered, that RunListeners can't really=
<br>
fail tests because fail() throws an AssertionException and Exceptions <br>
remove the RunListener. Is there a better way to discover output?<br>
<br>
To summarize what I want:<br>
* a way to discover output via System.out without preventing Logger <br>
functionality<br>
* this way should not involve huge code changes on the testclasses ( <br>
like adding TestRules everywhere)<br>
* all unittests which produced output should be reported as "failing&q=
uot; <br>
when you run with `mvn test`<br>
<br>
I tried to answer these questions on SO but got only a few helpful <br>
responses, so I thought I ask the specialists here and get more <br>
opinions. For reference the SO questions:<br>
* <a href=3D"http://stackoverflow.com/questions/33214683" target=3D"_blank"=
>http://stackoverflow.com/questions/33214683</a><br>
* <a href=3D"http://stackoverflow.com/questions/33207996" target=3D"_blank"=
>http://stackoverflow.com/questions/33207996</a><br>
<br>
Any help on this topic is very appreciated.<br>
<br>
Regards<br>
Andreas Worm<br>
<br>
</p>
</div>
=20=20=20=20=20
=20=20=20=20
<div style=3D"color:#fff;min-height:0"></div>
</div>
=20=20
</blockquote></div><br></div>
<!-- |**|begin egp html banner|**| -->
<br>
<br>
<!-- |**|end egp html banner|**| -->
<div width=3D"1" style=3D"color: white; clear: both;"/>__._,_.___</div>
=20=20=20=20=20=20
=20=20
=20=20=20=20
<div id=3D"fromDMARC" style=3D"clear:both; margin-top: 10px;">
<hr style=3D"height:2px ; border-width:0; color:#E3E3E3; backgroun=
d-color:#E3E3E3;">
Posted by: Dawid Weiss <[email protected]> <hr s=
tyle=3D"height:2px ; border-width:0; color:#E3E3E3; background-color:#E3E3E=
3;">
</div>
<!-- Start Recommendations -->
<!-- End Recommendations -->
<!-- |**|begin egp html banner|**| -->
<img src=3D"http://geo.yahoo.com/serv?s=3D97476590/grpId=3D2423328/grpspI=
d=3D1705006905/msgId=3D24714/stime=3D1445495084" width=3D"1" height=3D"1"> =
<br>
<!-- |**|end egp html banner|**| -->
=20=20
<!-- |**|begin egp html banner|**| -->
<br>
=20=20=20
=20=20=20=20=20
=20
<!-- |**|begin egp html banner|**| -->
<div id=3D"ygrp-vital" style=3D"background-color: #f2f2f2; font-fam=
ily: Verdana; font-size: 10px; margin-bottom: 10px; padding: 10px;">
<span id=3D"vithd" style=3D"font-weight: bold; color: #333; text-tr=
ansform: uppercase; "><a href=3D"https://groups.yahoo.com/neo/groups/junit/=
info;_ylc=3DX3oDMTJlNGFxbjNqBF9TAzk3MzU5NzE0BGdycElkAzI0MjMzMjgEZ3Jwc3BJZAM=
xNzA1MDA2OTA1BHNlYwN2dGwEc2xrA3ZnaHAEc3RpbWUDMTQ0NTQ5NTA4NA--" style=3D"tex=
t-decoration: none;">Visit Your Group</a></span>
<ul style=3D"list-style-type: none; margin: 0; padding: 0; display: in=
line;">
<li style=3D"border-right: 1px solid #000; font-weight: 700; di=
splay: inline; padding: 0 5px; margin-left: 0;">
<span class=3D"cat"><a href=3D"https://groups.yahoo.com/neo/groups/ju=
nit/members/all;_ylc=3DX3oDMTJmM2gyYmo4BF9TAzk3MzU5NzE0BGdycElkAzI0MjMzMjgE=
Z3Jwc3BJZAMxNzA1MDA2OTA1BHNlYwN2dGwEc2xrA3ZtYnJzBHN0aW1lAzE0NDU0OTUwODQ-" s=
tyle=3D"text-decoration: none;">New Members</a></span>
<span class=3D"ct" style=3D"color: #ff7900;">3</span>
</li>
</ul>
</div>
<div id=3D"ft" style=3D"font-family: Arial; font-size: 11px; margin-top: 5p=
x; padding: 0 2px 0 0; clear: both;">
<a href=3D"https://groups.yahoo.com/neo;_ylc=3DX3oDMTJkbGoybmRxBF9TAzk3ND=
c2NTkwBGdycElkAzI0MjMzMjgEZ3Jwc3BJZAMxNzA1MDA2OTA1BHNlYwNmdHIEc2xrA2dmcARzd=
GltZQMxNDQ1NDk1MDg0" style=3D"float: left;"><img src=3D"http://l.yimg.com/r=
u/static/images/yg/img/email/new_logo/logo-groups-137x15.png" height=3D"15"=
width=3D"137" alt=3D"Yahoo! Groups" style=3D"border: 0;"/></a>
<div style=3D"color: #747575; float: right;"> • <a href=3D"https://i=
nfo.yahoo.com/privacy/us/yahoo/groups/details.html" style=3D"text-decoratio=
n: none;">Privacy</a> • <a href=3D"mailto:junit-unsubscribe@yahoogroup=
s.com?subject=3DUnsubscribe" style=3D"text-decoration: none;">Unsubscribe</=
a> • <a href=3D"https://info.yahoo.com/legal/us/yahoo/utos/terms/" sty=
le=3D"text-decoration: none;">Terms of Use</a> </div>
</div>
<!-- |**|end egp html banner|**| -->
</div> <!-- ygrp-msg -->
=20=20=20
<br>
<!-- |**|end egp html banner|**| -->
<div style=3D"color: white; clear: both;"/>__,_._,___</div>
</body>
</html>
--001a113f9070ed3b340522ab8c2d--