Re: junit 4 - generating a test suite at runtime for each pair of tests.

"Ray Tayek [email protected] [junit]" <[email protected]> Mon, 29 Jun 2015 23:34:41 -0700
Newsgroups gmane.comp.java.junit.user
Message-ID <[email protected]>
--------------000208010907010000010005
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit

On 6/24/2015 6:56 AM, David Saff [email protected] [junit] wrote:
> I'm with Kevin Cooney here; there likely is a way to get the pair-wise 
> suite you're asking for, but for the problem of "find the pairing 
> that's causing the problem as quickly as possible", binary search with 
> @Ignore does seem highly likely to be the answer.
very tedious if you have many tests.

thanks
>
> On Tue, Jun 23, 2015 at 7:45 PM, Ray Tayek [email protected] 
> <mailto:[email protected]> [junit] <[email protected] 
> <mailto:[email protected]>> wrote:
>
>     On 6/23/2015 3:44 AM, Paul King [email protected]
>     <mailto:[email protected]> [junit] wrote:
>>
>>
>>     I would guess that he wants the pair testing to help track down
>>     "global state" problem
>>     rather than as a permanent measure.
>>
>     yes. i have many tests that use sockets and threads.
>
>     i was hoping to get them by using: @RunWith(Suite.class)
>     @Suite.SuiteClasses({PairsTestCase.class}) public class
>     RunWithSuiteTextCase {] and some kind of filtering, but i am
>     getting lost.
>
>     thanks
>
>>
>>     On 23/06/2015 7:07 PM, Colin Vipurs [email protected]
>>     <mailto:[email protected]> [junit] wrote:
>>     > Hi Ray,
>>     >
>>     > It sounds like one (or more) of your tests is manipulating
>>     global state which is affecting one (or more) other tests, hence
>>     the reason for the failing test passing when run by itself.
>>     Rather than generating a combination of tests it would be much
>>     better to isolate the state manipulation so that it can be
>>     controlled correctly from within the test.
>>     >
>>     > On Tue, Jun 23, 2015 at 2:49 AM, Ray Tayek [email protected]
>>     <mailto:[email protected]> <mailto:[email protected]>
>>     <mailto:[email protected]> [junit] <[email protected]
>>     <mailto:[email protected]> <mailto:[email protected]>
>>     <mailto:[email protected]>> wrote:
>>     >
>>     > __
>>     >
>>     > using junit 4, i have a test case
>>     <http://xunitpatterns.com/Testcase%20Class.html>
>>     <http://xunitpatterns.com/Testcase%20Class.html> that has n test
>>     methods <http://xunitpatterns.com/Test%20Method.html>
>>     <http://xunitpatterns.com/Test%20Method.html>, one of the test
>>     methods fails when the test case is run. and this failing test
>>     method succeeds when run by itself.
>>     >
>>     > i would like to dynamically (at run time) create a set of test
>>     suites <http://xunitpatterns.com/Test%20Suite%20Object.html>
>>     <http://xunitpatterns.com/Test%20Suite%20Object.html> (or test
>>     cases), one for each pair of test methods in the test case.
>>     >
>>     > so a test case with 3 test methods would generate 3!=6 test
>>     suites (or test cases).
>>     >
>>     > thanks
>>     >
>>     > --
>>     > Honesty is a very expensive gift. So, don't expect it from
>>     cheap people - Warren Buffett
>>     > http://tayek.com/
>>     >
>>     >
>>     >
>>     >
>>     > --
>>     > Maybe she awoke to see the roommate's boyfriend swinging from
>>     the chandelier wearing a boar's head.
>>     >
>>     > Something which you, I, and everyone else would call "Tuesday",
>>     of course.
>>     >
>>     >
>>
>>     ---
>>     This email has been checked for viruses by Avast antivirus software.
>>     https://www.avast.com/antivirus
>>
>
>
>     -- 
>     Honesty is a very expensive gift. So, don't expect it from cheap people - Warren Buffett
>     http://tayek.com/
>
>
> 


-- 
Honesty is a very expensive gift. So, don't expect it from cheap people - Warren Buffett
http://tayek.com/


--------------000208010907010000010005
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit





<head>

<style type="text/css">
<!--

/* start of attachment style */
       .ygrp-photo-title{
         clear: both;
         font-size: smaller;
         height: 15px;
         overflow: hidden;
         text-align: center;
         width: 75px;
       }
       div.ygrp-photo{
         background-position: center;
         background-repeat: no-repeat;
         background-color: white;
         border: 1px solid black;
         height: 62px;
         width: 62px;
       }

       div.photo-title 
         a,
         div.photo-title a:active,
         div.photo-title a:hover,
         div.photo-title a:visited {
           text-decoration: none; 
       }

       div.attach-table div.attach-row {
         clear: both;
       }

       div.attach-table div.attach-row div {
         float: left;
         /* margin: 2px;*/
       }

       p {
         clear: both;
         padding: 15px 0 3px 0;
	 overflow: hidden;
       }

       div.ygrp-file {
         width: 30px;
         valign: middle;
       }
       div.attach-table div.attach-row div div a {
         text-decoration: none;
       }

       div.attach-table div.attach-row div div span {
         font-weight: normal;
       }

       div.ygrp-file-title {
         font-weight: bold;
       }
 /* end of attachment style */
        -->
        </style>
        </head>
<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">


<!-- |**|begin egp html banner|**| -->

<br><br>

<!-- |**|end egp html banner|**| -->



    <div class="moz-cite-prefix">On 6/24/2015 6:56 AM, David Saff
      <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> [junit] wrote:<br>
    </div>
    <blockquote
cite="mid:CALrw-Pz31Y52NL7UW8-d5iNONv9NBDu9wApJGBk4o5ret9qfUA@mail.gmail.com"
      type="cite"> <span style="display:none"> </span>
      
          <div id="ygrp-text">
            <div dir="ltr">I'm with Kevin Cooney here; there likely is a
              way to get the pair-wise suite you're asking for, but for
              the problem of "find the pairing that's causing the
              problem as quickly as possible", binary search with
              @Ignore does seem highly likely to be the answer.</div>
          </div>
        </div>
      </div>
    </blockquote>
    very tedious if you have many tests. <br>
    <br>
    thanks<br>
    <blockquote
cite="mid:CALrw-Pz31Y52NL7UW8-d5iNONv9NBDu9wApJGBk4o5ret9qfUA@mail.gmail.com"
      type="cite">
      <div id="ygrp-mlmsg" style="position:relative;">
        <div id="ygrp-msg" style="z-index: 1;">
          <div id="ygrp-text">
            <div class="gmail_extra"><br>
              <div class="gmail_quote">On Tue, Jun 23, 2015 at 7:45 PM,
                Ray Tayek <a moz-do-not-send="true"
                  href="mailto:[email protected]">[email protected]</a>
                [junit] <span dir="ltr">&lt;<a moz-do-not-send="true"
                    href="mailto:[email protected]" target="_blank">[email protected]</a>&gt;</span>
                wrote:<br>
                <blockquote class="gmail_quote" style="border-left:1px
                  #ccc solid;">
                  <div style="background-color:#fff;">
                    <span> </span>
                    <div>
                      <div>
                        <div>
                          <p><span class=""> </span></p>
                          <div>On 6/23/2015 3:44 AM, Paul King <a
                              moz-do-not-send="true"
                              href="mailto:[email protected]"
                              target="_blank">[email protected]</a>
                            [junit] wrote:<br>
                          </div>
                          <blockquote type="cite"> <span> </span>
                            <div>
                              <p><br>
                                I would guess that he wants the pair
                                testing to help track down "global
                                state" problem<br>
                                rather than as a permanent measure.<br>
                              </p>
                            </div>
                          </blockquote>
                          yes. i have many tests that use sockets and
                          threads.<br>
                          <br>
                          i was hoping to get them by using:
                          @RunWith(Suite.class)
                          @Suite.SuiteClasses({PairsTestCase.class})
                          public class RunWithSuiteTextCase {] and some
                          kind of filtering, but i am getting lost.<br>
                          <br>
                          thanks<span class=""><br>
                            <br>
                            <blockquote type="cite">
                              <div>
                                <div>
                                  <div>
                                    <p> <br>
                                      On 23/06/2015 7:07 PM, Colin
                                      Vipurs <a moz-do-not-send="true"
href="mailto:[email protected]" target="_blank">[email protected]</a>
                                      [junit] wrote:<br>
                                      &gt; Hi Ray,<br>
                                      &gt;<br>
                                      &gt; It sounds like one (or more)
                                      of your tests is manipulating
                                      global state which is affecting
                                      one (or more) other tests, hence
                                      the reason for the failing test
                                      passing when run by itself. Rather
                                      than generating a combination of
                                      tests it would be much better to
                                      isolate the state manipulation so
                                      that it can be controlled
                                      correctly from within the test.<br>
                                      &gt;<br>
                                      &gt; On Tue, Jun 23, 2015 at 2:49
                                      AM, Ray Tayek <a
                                        moz-do-not-send="true"
                                        href="mailto:[email protected]"
                                        target="_blank">[email protected]</a>
                                      <a moz-do-not-send="true"
                                        href="mailto:[email protected]"
                                        target="_blank">&lt;mailto:[email protected]&gt;</a>
                                      [junit] &lt;<a
                                        moz-do-not-send="true"
                                        href="mailto:[email protected]"
                                        target="_blank">[email protected]</a>
                                      <a moz-do-not-send="true"
                                        href="mailto:[email protected]"
                                        target="_blank">&lt;mailto:[email protected]&gt;</a>&gt;
                                      wrote:<br>
                                      &gt;<br>
                                      &gt; __<br>
                                      &gt;<br>
                                      &gt; using junit 4, i have a test
                                      case <a moz-do-not-send="true"
                                        href="http://xunitpatterns.com/Testcase%20Class.html"
                                        target="_blank">&lt;http://xunitpatterns.com/Testcase%20Class.html&gt;</a>
                                      that has n test methods <a
                                        moz-do-not-send="true"
                                        href="http://xunitpatterns.com/Test%20Method.html"
                                        target="_blank">&lt;http://xunitpatterns.com/Test%20Method.html&gt;</a>,
                                      one of the test methods fails when
                                      the test case is run. and this
                                      failing test method succeeds when
                                      run by itself.<br>
                                      &gt;<br>
                                      &gt; i would like to dynamically
                                      (at run time) create a set of test
                                      suites <a moz-do-not-send="true"
href="http://xunitpatterns.com/Test%20Suite%20Object.html"
                                        target="_blank">&lt;http://xunitpatterns.com/Test%20Suite%20Object.html&gt;</a>
                                      (or test cases), one for each pair
                                      of test methods in the test case.<br>
                                      &gt;<br>
                                      &gt; so a test case with 3 test
                                      methods would generate 3!=6 test
                                      suites (or test cases).<br>
                                      &gt;<br>
                                      &gt; thanks<br>
                                      &gt;<br>
                                      &gt; --<br>
                                      &gt; Honesty is a very expensive
                                      gift. So, don't expect it from
                                      cheap people - Warren Buffett<br>
                                      &gt; <a moz-do-not-send="true"
                                        href="http://tayek.com/"
                                        target="_blank">http://tayek.com/</a><br>
                                      &gt;<br>
                                      &gt;<br>
                                      &gt;<br>
                                      &gt;<br>
                                      &gt; --<br>
                                      &gt; Maybe she awoke to see the
                                      roommate's boyfriend swinging from
                                      the chandelier wearing a boar's
                                      head.<br>
                                      &gt;<br>
                                      &gt; Something which you, I, and
                                      everyone else would call
                                      "Tuesday", of course.<br>
                                      &gt;<br>
                                      &gt; <br>
                                      <br>
                                      ---<br>
                                      This email has been checked for
                                      viruses by Avast antivirus
                                      software.<br>
                                      <a moz-do-not-send="true"
                                        href="https://www.avast.com/antivirus"
                                        target="_blank">https://www.avast.com/antivirus</a><br>
                                      <br>
                                    </p>
                                  </div>
                                </div>
                              </div>
                            </blockquote>
                            <br>
                            <br>
                            <pre cols="72">-- 
Honesty is a very expensive gift. So, don't expect it from cheap people - Warren Buffett
<a moz-do-not-send="true" href="http://tayek.com/" target="_blank">http://tayek.com/</a></pre>
                          </span> </div>
                      </div>
                    </div>
                  </div>
                </blockquote>
              </div>
              <br>
            </div>
          </div>
          
      
      <!-- end group email -->
    </blockquote>
    <br>
    <br>
    <pre class="moz-signature" cols="72">-- 
Honesty is a very expensive gift. So, don't expect it from cheap people - Warren Buffett
<a class="moz-txt-link-freetext" href="http://tayek.com/">http://tayek.com/</a></pre>
  



<!-- |**|begin egp html banner|**| -->

<br>


<br>

<!-- |**|end egp html banner|**| -->


<div width="1" style="color: white; clear: both;"/>__._,_.___</div>

      
  

    
    <div id="fromDMARC" style="clear:both; margin-top: 10px;">
         <hr style="height:2px ; border-width:0; color:#E3E3E3; background-color:#E3E3E3;">
         Posted by: Ray Tayek &lt;[email protected]&gt;         <hr style="height:2px ; border-width:0; color:#E3E3E3; background-color:#E3E3E3;">
    </div>
<!-- Start Recommendations -->
<!-- End Recommendations -->



<!-- |**|begin egp html banner|**| -->

  <img src="http://geo.yahoo.com/serv?s=97476590/grpId=2423328/grpspId=1705006905/msgId=24677/stime=1435646281" width="1" height="1"> <br>

<!-- |**|end egp html banner|**| -->

    
<!-- |**|begin egp html banner|**| -->

<br>



   
     
 
        <!-- |**|begin egp html banner|**| -->
        <div id="ygrp-vital" style="background-color: #f2f2f2; font-family: Verdana; font-size: 10px; margin-bottom: 10px; padding: 10px;">

        <span id="vithd" style="font-weight: bold; color: #333; text-transform: uppercase; "><a href="https://groups.yahoo.com/neo/groups/junit/info;_ylc=X3oDMTJlOTdhaTh1BF9TAzk3MzU5NzE0BGdycElkAzI0MjMzMjgEZ3Jwc3BJZAMxNzA1MDA2OTA1BHNlYwN2dGwEc2xrA3ZnaHAEc3RpbWUDMTQzNTY0NjI4MQ--" style="text-decoration: none;">Visit Your Group</a></span>

     <ul style="list-style-type: none; margin: 0; padding: 0; display: inline;">
            <li style="border-right: 1px solid #000; font-weight: 700; display: inline; padding: 0 5px; margin-left: 0;">
      <span class="cat"><a href="https://groups.yahoo.com/neo/groups/junit/members/all;_ylc=X3oDMTJmZHNoNzFnBF9TAzk3MzU5NzE0BGdycElkAzI0MjMzMjgEZ3Jwc3BJZAMxNzA1MDA2OTA1BHNlYwN2dGwEc2xrA3ZtYnJzBHN0aW1lAzE0MzU2NDYyODE-" style="text-decoration: none;">New Members</a></span>
      <span class="ct" style="color: #ff7900;">2</span>
    </li>
                                              </ul>
  </div>


<div id="ft" style="font-family: Arial; font-size: 11px; margin-top: 5px; padding: 0 2px 0 0; clear: both;">
  <a href="https://groups.yahoo.com/neo;_ylc=X3oDMTJkYWxmM2RsBF9TAzk3NDc2NTkwBGdycElkAzI0MjMzMjgEZ3Jwc3BJZAMxNzA1MDA2OTA1BHNlYwNmdHIEc2xrA2dmcARzdGltZQMxNDM1NjQ2Mjgx" style="float: left;"><img src="http://l.yimg.com/ru/static/images/yg/img/email/new_logo/logo-groups-137x15.png" height="15" width="137" alt="Yahoo! Groups" style="border: 0;"/></a>
  <div style="color: #747575; float: right;"> &bull; <a href="https://info.yahoo.com/privacy/us/yahoo/groups/details.html" style="text-decoration: none;">Privacy</a> &bull; <a href="mailto:[email protected]?subject=Unsubscribe" style="text-decoration: none;">Unsubscribe</a> &bull; <a href="https://info.yahoo.com/legal/us/yahoo/utos/terms/" style="text-decoration: none;">Terms of Use</a> </div>
</div>

<!-- |**|end egp html banner|**| -->

  </div> <!-- ygrp-msg -->

   


  <br>

<!-- |**|end egp html banner|**| -->


<div style="color: white; clear: both;"/>__,_._,___</div>
</body>
</html>

--------------000208010907010000010005--