Timed Tests and Thread Safety

Hi All,

I've been using timed tests (i.e. @Test(timeout = ...)) for years but recently came across an implementation detail of which I was unaware.  While I knew that timed tests are run in a thread other than the main test thread, I assumed that all aspects of the test (set up, test, and tear down) were run in that other thread.  I was surprised to discover that the set up and tear down methods are run on the main test thread, while the test method is run on a different thread.

This jumped out at me in a recent programming session because the class under test uses thread isolation to achieve thread safety, and its methods liberally assert that they are invoked on the same thread on which the object was instantiated.  Once I realized why my timed tests were failing unexpectedly, I worked around the problem by moving the necessary portions of my set up and tear down to the test method itself.

However, this got me thinking about more subtle issues related to timed tests of code that is inherently NOT thread safe.  Consider the following contrived example, where Bar is the class under test:

public class Foo {
    public void dispose() {
    }
}

public class Bar {
    private Foo foo = null;

    public void dispose() {
        if (foo != null) {
            foo.dispose();
            foo = null;
        }
    }

    public void setFoo(Foo foo) {
        this.foo = foo;
    }
}

public class BarTest {
    private Bar bar;

    @Before
    public void setUp() {
        bar  = new Bar();
    }

    @Test
    public void testSomethingOnMainThread() {
        // ...
        bar.setFoo(new Foo());
        // ...
    }

    @Test(timeout = 1000)
    public void testSomethingOnDifferentThread() {
        // ...
        bar.setFoo(new Foo());
        // ...
    }

    @After
    public void tearDown() {
        bar.dispose();
    }
}

When testSomethingOnMainThread() is run, I expect the Foo instance created in the test will have its dispose() method called in tearDown() because its assignment to the Bar.foo field occurs on the same thread that invokes tearDown().

However, when testSomethingOnDifferentThread() is run, I expect the Foo instance created in the test MAY NOT have its dispose() method called in tearDown() because its assignment to the Bar.foo field occurs on a different thread than the one that invokes tearDown().  Without some kind of memory barrier, the Bar.foo field may not be safely published from the test thread to the main thread.

My concern is that there could be a significant change in the behavior of a test simply by annotating it with a timeout when the code under test is not thread safe.  I can't be the first person to ask this question (I apologize  if my search of this group's archives was not thorough enough), so I realize my entire premise may be ill-posed or that I'm missing some fundamental understanding of the Java memory model.

With that said, is it reasonable to expect that, for a given @Test, all @Before and @After methods in the fixture be called on the same thread regardless of how the test is annotated?  Or is it simply up to the programmer to be aware of how timed tests are implemented, and to make it their responsibility to ensure code that is not thread safe is only ever invoked on a single thread, or to use external synchronization to ensure safe publication between threads?

Cheers,
Steve

[Non-text portions of this message have been removed]



------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> Your email settings:
    Individual Email | Traditional

<*> To change settings online go to:
    http://groups.yahoo.com/group/junit/join
    (Yahoo! ID required)

<*> To change settings via email:
    [email protected] 
    [email protected]

<*> To unsubscribe from this group, send an email to:
    [email protected]

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/