JUnit compatibility (was: Re: Re: JUnit 4.12-beta-1 is released)

Stefan,

Thanks for your mail.  The original thread is indeed about whether a
particular release has breaking changes, an important topic, so I forked a
new thread to address the more general questions you raise.

In general, JUnit 4 has indeed striven for semantic versioning.  Since
we've been on 4.x for some 9-ish years now, that's meant trying to maintain
backwards compatibility across a wide codebase, sometimes at considerable
cost to effort and design coherence, but it's still been a primary goal of
my involvement.  I'm not sure what it would mean to have an "explicit
compatibility policy"; if there's another open-source project that has a
great posted policy, that's something I'd consider.

The two historical examples you speak of are both mistakes.  The change
from JUnit 4.4 to 4.5 was the movement of AssumptionViolatedException,
unless there's another change you have in mind (
https://jira.spring.io/browse/SPR-5145).  Issue 357 was a mess that I
caused.  I hope we can avoid making such mistakes going forward, which is
why I'm glad that this latest beta surfaced the JUnit4ClassRunner
incompatibility.

   David Saff

On Tue, Aug 12, 2014 at 5:18 PM, Stefan [email protected] [junit] <
[email protected]> wrote:

>
>
> Hi,
>
> I agree with Arie that the whole discussion should turn from a discussion
> about backward compatibility of a release to a discussion about a
> compatibility policy. In my opinion semantic versioning is not only about
> version numbers but about a clear compatibility policy that will be
> guaranteed by the authors of the library.
>
> As far as I know for JUnit there is no compatibility policy at all -
> neither formally agreed upon nor informal. On the one hand JUnit 4 ships
> with a JUnit 3 compatibility layer that will not be removed in order to
> keep "old legacy code bases alive". On the other hand there have been
> incompatible changes in JUnit minor versions that forced test-support
> library developers and excessive JUnit users to adopt. For example the
> runner structure changed between 4.4 and 4.5 causing projects like spring
> to develop workarounds and reflection magic to support both versions. From
> 4.9 to 4.10 test rule implementation was changed (see
> https://github.com/junit-team/junit/issues/357) and there might be more
> examples I'm not aware of.
>
> So why do you maintain classes from a version that had it's end of life 8
> years ago to keep tests running written more than 8 years ago while
> breaking code that has been developed months or days ago?
>
> Personally I did not suffer from the breaking changes in 4.5 nor will I
> suffer from the planned changes in 4.12 - but the test rule implementation
> change in 4.10 was a lot of effort for me and some of my collegues (see
> github issue for details).
> 4.10 was released in 2011 and I had to maintain two versions of our
> test-support library to support projects using JUnit 4.8 and 4.9 on the one
> hand and newer or updated projects using JUnit 4.10 and 4.11 on the other.
> Some months ago our last active developed project migrated from JUnit 4.8.2
> to JUnit 4.11 and I could finally stop supporting and developing the old
> branch of the test-support library.
>
>
> To sum up: I'd love to have a compatibility policy for JUnit that every
> one can rely on. Semantic versioning would be a good choice.
>
> Regards,
> Stefan
>
>
> Am 12.08.2014 um 21:06 schrieb David Saff [email protected] [junit]:
>
>
> Henrik,
>
>  Thanks for the update.  We're considering a solution very close to what
> you recommend below; I'll report back.
>
>     David
>
>
>  On Tue, Aug 12, 2014 at 1:53 PM, [email protected] [junit] <
> [email protected]> wrote:
>
>>
>>
>> Hi DavidI have tried 4.12-beta-1 on a number of problem scenarios. There
>> will be some problems with publicly available 3rd-party runners but the
>> most serious consequences (for end-users) will probably show up where they
>> have some customized in-house runner ...public class
>> Somebodies_Inhouse_Runner extends JUnit4ClassRunner {  public
>> Someones_Inhourse_Runner(Class<?> testClass) {super(testClass);}}... and a
>> common test super-class ...@ RunWith(Somebodies_Inhouse_Runner.class)public
>> abstract TestSuperClass { /* Common test-utils go here! */ }... both of
>> which class-files are available on the test class-path (perhaps in a
>> separate jar-file).A test that extends TestSuperClass ...public class
>> BrokenTest extends TestSuperClass {  @ Test  public void myTestMethod()
>> {}}... will compile and run as expected with JUnit-4.11. With 4.12-beta-1
>> it will still compile (without warnings!) but the test-run will result in
>> this error message:"Type [unknown] not present"(I have verified this
>> scenario with JDK-7)My opinion on this:Keeping JUnit4ClassRunner - and
>> finalize its deprecation by having its constructor throw an understandable
>> error-message - would leave the JUnit community with a maintenance burden
>> that is insignificant compared to the burden of finding out the meaning of
>> "Type [unknown] not present"!BR /Henrik Kaipe
>>
>
>
>  
>