Re: Re: JUnit 4.12-beta-1 is released

Hi,

I agree with Arie that the whole discussion should turn from a 
discussion about backward compatibility of a release to a discussion 
about a compatibility policy. In my opinion semantic versioning is not 
only about version numbers but about a clear compatibility policy that 
will be guaranteed by the authors of the library.

As far as I know for JUnit there is no compatibility policy at all - 
neither formally agreed upon nor informal. On the one hand JUnit 4 ships 
with a JUnit 3 compatibility layer that will not be removed in order to 
keep "old legacy code bases alive". On the other hand there have been 
incompatible changes in JUnit minor versions that forced test-support 
library developers and excessive JUnit users to adopt. For example the 
runner structure changed between 4.4 and 4.5 causing projects like 
spring to develop workarounds and reflection magic to support both 
versions. From 4.9 to 4.10 test rule implementation was changed (see 
https://github.com/junit-team/junit/issues/357) and there might be more 
examples I'm not aware of.

So why do you maintain classes from a version that had it's end of life 
8 years ago to keep tests running written more than 8 years ago while 
breaking code that has been developed months or days ago?

Personally I did not suffer from the breaking changes in 4.5 nor will I 
suffer from the planned changes in 4.12 - but the test rule 
implementation change in 4.10 was a lot of effort for me and some of my 
collegues (see github issue for details).
4.10 was released in 2011 and I had to maintain two versions of our 
test-support library to support projects using JUnit 4.8 and 4.9 on the 
one hand and newer or updated projects using JUnit 4.10 and 4.11 on the 
other. Some months ago our last active developed project migrated from 
JUnit 4.8.2 to JUnit 4.11 and I could finally stop supporting and 
developing the old branch of the test-support library.


To sum up: I'd love to have a compatibility policy for JUnit that every 
one can rely on. Semantic versioning would be a good choice.

Regards,
Stefan


Am 12.08.2014 um 21:06 schrieb David Saff [email protected] [junit]:
> Henrik,
>
> Thanks for the update.  We're considering a solution very close to 
> what you recommend below; I'll report back.
>
>  David
>
>
> On Tue, Aug 12, 2014 at 1:53 PM, [email protected] 
> <mailto:[email protected]> [junit] <[email protected] 
> <mailto:[email protected]>> wrote:
>
>     Hi DavidI have tried 4.12-beta-1 on a number of problem scenarios.
>     There will be some problems with publicly available 3rd-party
>     runners but the most serious consequences (for end-users) will
>     probably show up where they have some customized in-house runner
>     ...public class Somebodies_Inhouse_Runner extends
>     JUnit4ClassRunner {  public Someones_Inhourse_Runner(Class<?>
>     testClass) {super(testClass);}}... and a common test super-class
>     ...@ RunWith(Somebodies_Inhouse_Runner.class)public abstract
>     TestSuperClass { /* Common test-utils go here! */ }... both of
>     which class-files are available on the test class-path (perhaps in
>     a separate jar-file).A test that extends TestSuperClass ...public
>     class BrokenTest extends TestSuperClass {  @ Test  public void
>     myTestMethod() {}}... will compile and run as expected with
>     JUnit-4.11. With 4.12-beta-1 it will still compile (without
>     warnings!) but the test-run will result in this error
>     message:"Type [unknown] not present"(I have verified this scenario
>     with JDK-7)My opinion on this:Keeping JUnit4ClassRunner - and
>     finalize its deprecation by having its constructor throw an
>     understandable error-message - would leave the JUnit community
>     with a maintenance burden that is insignificant compared to the
>     burden of finding out the meaning of "Type [unknown] not
>     present"!BR /Henrik Kaipe
>
>
>