Re: Timed Tests and Thread Safety

--- In [email protected], Kevin Cooney <kcooney@...> wrote:
> Note the Timeout rule will call @Before and @After methods in the
> same thread as the test. The test class is created before rules are
> applied, so if your test class initializes fields at declaration
> time, that execution would happen in a different thread than the
> test.
> 
> I also believe that @After methods would not be called for tests
> methods that timeout in classes that use the Timeout rule.

@Kevin: I feel like a fool.  I actually "discovered" rules during this exercise and spent some time changing legacy @Test(timeout = ...) tests to use a fixture-level Timeout @Rule.  However, I just assumed that the threading behavior between the two was the same. :(  Based on your information, I ran some more tests to nail down on which thread the various parts of the fixture are executed using both models.  The results are summarized below (from JUnit 4.8.2).

    @Test with timeout element:
        A1. ctor run on main thread
        A2. @BeforeClass, @Before, @After, @AfterClass run on main thread
        A3. @Test run on worker thread
        A4. @After, @AfterClass run on main thread if test times out

    Timeout @Rule:
        B1. ctor run on main thread
        B2. @BeforeClass, @AfterClass run on main thread
        B3. @Before, @Test, @After run on worker thread
        B4. @After NOT run if test times out
        B5. @AfterClass run on main thread if test times out

Given these results, using Timeout @Rules satisfies my concerns about thread safety and gives me even more motivation to use them over @Test(timeout = ...).  Thanks!  (Note sure if item [B4] is intentional, but I can live with it for now.)


--- In [email protected], David Saff <saff@...> wrote:
> The biggest payoff would probably be to make sure that the
> documentation we have makes it easy to avoid any unexpected
> behavior.  Can you suggest an edit to that effect?

@David: Your suggestion for updating the documentation is probably the best solution based on the preferred behavior of the Timeout @Rule.  Below is my cut at a second paragraph I'd add to the Javadocs for the @Test annotation's timeout element.  Please edit as you see fit.

    THREAD SAFETY WARNING.  Test methods with a timeout parameter are run in a thread other than the thread which runs the fixture's @Before and @After methods.  This may yield different behavior for code that is not thread safe when compared to the same test method without a timeout parameter.  Consider using the org.junit.rules.Timeout rule instead, which ensures a test method is run on the same thread as the fixture's @Before and @After methods.

Thanks again for all the feedback,
Steve




------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/junit/join
    (Yahoo! ID required)

<*> To change settings via email:
    [email protected] 
    [email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/