Re: Timed Tests and Thread Safety

Steve,

You raise a great question.  Unfortunately, and perhaps I'm missing
something obvious, I don't think there's a way to provide timeout
semantics, guarantee that @After is called, and guarantee that @After is
called on the same thread as the @Test method.  Essentially, once the
runner hands over control to the @Test thread, there's no guarantee that it
will ever return control back--it could go into an infinite loop that
catches InterruptedExceptions.

It would potentially be possible to guarantee that successful tests that
complete within the timeout have @After methods called on the same thread,
but that would further complicate matters, with questionable (IMHO) payoff.

The biggest payoff would probably be to make sure that the documentation we
have makes it easy to avoid any unexpected behavior.  Can you suggest an
edit to that effect?  Thanks,

   David Saff


On Thu, May 23, 2013 at 8:34 PM, Steven Soloff <[email protected]>wrote:

> **
>
>
> Hi All,
>
> I've been using timed tests (i.e. @Test(timeout = ...)) for years but
> recently came across an implementation detail of which I was unaware. While
> I knew that timed tests are run in a thread other than the main test
> thread, I assumed that all aspects of the test (set up, test, and tear
> down) were run in that other thread. I was surprised to discover that the
> set up and tear down methods are run on the main test thread, while the
> test method is run on a different thread.
>
> This jumped out at me in a recent programming session because the class
> under test uses thread isolation to achieve thread safety, and its methods
> liberally assert that they are invoked on the same thread on which the
> object was instantiated. Once I realized why my timed tests were failing
> unexpectedly, I worked around the problem by moving the necessary portions
> of my set up and tear down to the test method itself.
>
> However, this got me thinking about more subtle issues related to timed
> tests of code that is inherently NOT thread safe. Consider the following
> contrived example, where Bar is the class under test:
>
> public class Foo {
> public void dispose() {
> }
> }
>
> public class Bar {
> private Foo foo = null;
>
> public void dispose() {
> if (foo != null) {
> foo.dispose();
> foo = null;
> }
> }
>
> public void setFoo(Foo foo) {
> this.foo = foo;
> }
> }
>
> public class BarTest {
> private Bar bar;
>
> @Before
> public void setUp() {
> bar = new Bar();
> }
>
> @Test
> public void testSomethingOnMainThread() {
> // ...
> bar.setFoo(new Foo());
> // ...
> }
>
> @Test(timeout = 1000)
> public void testSomethingOnDifferentThread() {
> // ...
> bar.setFoo(new Foo());
> // ...
> }
>
> @After
> public void tearDown() {
> bar.dispose();
> }
> }
>
> When testSomethingOnMainThread() is run, I expect the Foo instance created
> in the test will have its dispose() method called in tearDown() because its
> assignment to the Bar.foo field occurs on the same thread that invokes
> tearDown().
>
> However, when testSomethingOnDifferentThread() is run, I expect the Foo
> instance created in the test MAY NOT have its dispose() method called in
> tearDown() because its assignment to the Bar.foo field occurs on a
> different thread than the one that invokes tearDown(). Without some kind of
> memory barrier, the Bar.foo field may not be safely published from the test
> thread to the main thread.
>
> My concern is that there could be a significant change in the behavior of
> a test simply by annotating it with a timeout when the code under test is
> not thread safe. I can't be the first person to ask this question (I
> apologize if my search of this group's archives was not thorough enough),
> so I realize my entire premise may be ill-posed or that I'm missing some
> fundamental understanding of the Java memory model.
>
> With that said, is it reasonable to expect that, for a given @Test, all
> @Before and @After methods in the fixture be called on the same thread
> regardless of how the test is annotated? Or is it simply up to the
> programmer to be aware of how timed tests are implemented, and to make it
> their responsibility to ensure code that is not thread safe is only ever
> invoked on a single thread, or to use external synchronization to ensure
> safe publication between threads?
>
> Cheers,
> Steve
>
> [Non-text portions of this message have been removed]
>
>  
>


[Non-text portions of this message have been removed]



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/junit/join
    (Yahoo! ID required)

<*> To change settings via email:
    [email protected] 
    [email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/