Re: Decoupling JUnit and Hamcrest

Hi,

this seems to be a discussion from Stefan to Stefan, so I cant hesitate to post an answer. ;-)

I really think that separation is a good thing and should be enforced by the JUnit team. Hamcrest is a great framework for assertions, but in some cases it is not needed or not the best fit. So the test writers should have to possibility to choose the assertion framework of their choice and the testing framework should smoothly work with it.

What's behind my opinion? Some months ago I was playing around with different unit testing frameworks. While I was doing so I asked myself the question: How do you develop a unit testing framework using TDD practices? I sat down and started to create the first bits of code that would represent a "unit test". With this very simple test in place I wrote some production code to execute the test case. Fun it was! I tried some more tests and continued to play around like this for some hours and when I finished I realized that I had developed a very rudimental unit testing framework. While being very rudimental, the framework still supports things like:

- @Test, @Before, @After, @Ignore, @Context (like the hierarchical context runner for JUnit)
- Test Groups and Test Suites,
- Reporters (still no support for Listeners, but soon to come ;-))
- Extendable Tests Creators (you can register a Factory to support your own Test Cases, like cucumber files, etc.)

The core idea of the framework was not to create a new standard, but to play around with TDD for building a Unit Testing Framework. I came up with two guidelines, which I think are very important for a next generation of JUnit (5.x for example):

1. Provide a flexible and extendable framework that only cares about the execution and reporting of test cases and results
2. Don't struggle with things third party libraries can offer, i.e. don't try to be the one-and-only tool for TDD

In order to be flexible and extendable, I think there should be some kind of application context that third party libraries may register themselves for certain activities, e.g. finding test cases (no longer limited to annotated Java Classes, input can be anything like cucumber tests), providing assertions (I simply used Hamcrest in my tests, but the user of the framework should be free to choose anything that throws an AssertionError), or to run tests multiple times or expect exceptions (I used Java 8 lambdas to create a very well readable syntax for it). I finally felt like this is the way to go: Separate the core framework from things that can be easily provided separately.

To give a very short impression of what a test will look like using my framework, look at this test case:

// ...

public class TestContextTest {
  @Test
  public void givenInvalidTestContext_createInstanceFails() throws Exception {
    expectThrowable(AssertionError.class).withMessage("Class not instantiable").in(() -> {
      TestClass testClass = new TestClass(TestContextWithPrivateConstructor.class);
      TestContext testContext = new TestContext(testClass, TestContextWithPrivateConstructor.InvalidContext.class);
      testContext.createTestInstance();
    });
  }

  @Context
  public class GivenTestContextWithSingleTest {
    // ...

    @Before
    public void setupTestContext() throws Exception {
      parentInstance = new TestContextWithSingleTest();
      testClass = new TestClassMock(parentInstance);
      testContext = new TestContext(testClass, TestContextWithSingleTest.TestContext.class);
    }

    @Test
    public void createsAValidTestInstanceOfClassUnderTest() throws Exception {
      TestContextWithSingleTest.TestContext instance = testContext.createTestInstance();
      assertThat(instance, is(instanceOf(TestContextWithSingleTest.TestContext.class)));
    }
    
    // ...
  }

  // ...
}

Code taken from: https://github.com/bechte/JUT/blob/master/src/test/java/de/bechte/jut/testables/TestContextTest.java (I crunched some lines for readability reasons).

For those who are interested in more details, I uploaded JUT at github: https://github.com/bechte/JUT - I really think it is worth a look at and we might use some ideas from it for a future JUnit release (especially the lambda style expressions for exception handling, which I think is much more readable). :-)

All the best,

Stefan

Am 08.01.2015 um 23:15 schrieb Stefan Birkner [email protected] [junit] <[email protected]>:

> 
> Hi Stefan,
> 
> thank you for your feedback.
> 
> The JUnit team itself wants to get rid of the Hamcrest dependency. Basically Hamcrest is an assertion library (like AssertJ, Truth, ...) and it can be independent from JUnit. Therefore it must only use JUnit's interfaces (AssertionError and AssumptionViolatedException) and JUnit must not use Hamcrest. Both is possible, but it needs a new JUnit major release.
> 
> ciao,
> stefan
> 
> 2015-01-08 21:46 GMT+01:00 Stefan [email protected] [junit] <[email protected]>:
>  
> Hi,
> 
> I'm not a developer of junit nor hamcrest and I don't feel the pain of the tight integration of junit and hamcrest that "prevents both from moving on" but I'm a heavy user of both.
> I don't think it's a good idea to separate the hamcrest integration from junit into it's own library.
> 
> At the moment JUnit dictates the hamcrest version to be used but also guarantees compatibility with this version.
> With the new library hamcrest developers will dictate the junit version that is compatible with a specific hamcrest release. This is only a change in responsibility but has no benefits.
> 
> Of course there is also the opportunity to provide a broader range of compatible versions. But this opportunity also means more work for the maintainers and slower adoption by users. JUnit 4.12 uses hamcrest 1.3. What if JUnit moves on to 4.13, 4.14 and hamcrest releases 2.0 and 2.1 respectively. Will there be a version which is compatible with hamcrest 2.1 and JUnit 4.13? Will there be a version which is compatible with JUnit 4.14 and hamcrest 1.3?
> 
> As a developer who want's to use JUnit 4.14 - how do I know which hamcrest-junit-integration version to use? What will stop me from using a wrong version? For example I might be tempted to update JUnit to 4.14 but forget to update hamcrest-junit-integration. There might be changes of the behaviour of JUnit that might cause some of my tests to pass always although they should fail. On the other hand: JUnit 4.14 might be binary incompatible to JUnit 4.12 - where to look for the right version of hamcrest-junit-integration?
> 
> If hamcrest moves on to 2.0 - which could be binary incompatible to 1.3 due to the major version change (haven't checked) - how long will hamcrest 1.3 be supported with latest JUnit versions by hamcrest-junit-integration? I'm asking because we also use JMock which depends on hamcrest as well. If I update to an incompatible 2.0 I need also an update of JMock....
> (( Actually I cannot update JMock without big effort because there was an incompatible change from 2.5.1 to 2.6 .... ))
> 
> On the other hand this will slow down speed for adoption of new JUnit versions because we have to wait until a new hamcrest-junit-integration version is out or at least someone has built the appropriate version with latest JUnit release and verified unit tests are still passing. It will also increase the effort necessary to try JUnit release candidates or snapshots in real projects (because I also have to checkout and build the integration library myself to be sure there is no problem with that... and if there is a problem I need to fix it myself or stop using latest JUnit release candidate).
> 
> ----
> IMHO decoupling JUnit and hamcrest has more drawbacks than benefits. Although I see it can be a chance for hamcrest to move on. I don't think JUnit as a project will profit that much. JUnit users might have a broader choice of compatible hamcrest versions but this requires some effort (of the maintainers of hamcrest-junit-integration). I think this might only work well for JUnit users if the JUnit team commits to that integration project and is able to push out new releases of the hamcrest-junit-integration together with new JUnit releases.
> 
> 
> Regards,
> Stefan
> 
> 
> 
> Am 07.01.2015 um 08:31 schrieb Stefan Birkner [email protected] [junit]:
>>  
>> FYI: Nat Pryce took a first step for decoupling JUnit and Hamcrest:
>> http://natpryce.com/articles/000806.html
>> 
> 
> 
> 
> 
> 



[Non-text portions of this message have been removed]



------------------------------------
Posted by: Stefan Bechtold <[email protected]>
------------------------------------