Re: Quo Vadis JUnit

Oops, the group was set to require approval of new members.  I removed
that; come in and post anytime!

   David

On Thu, Oct 23, 2014 at 8:40 PM, David Saff <[email protected]> wrote:

> I think there's likely much more to talk about than could fit in a single
> thread.  Let's pick up the discussion here:
>
> https://groups.google.com/forum/#!forum/testing-java-8
>
> Anyone is welcome; within the group, we can start individual threads on
> individual subtopics, leaving this main list open for more incremental
> discussions about JUnit 4, and community troubleshooting.
>
>    David
>
> On Thu, Oct 23, 2014 at 5:12 PM, Stefan Bechtold [email protected]
> [junit] <[email protected]> wrote:
>
>> There are a lot of opinions, and great ideas for a new flexible testing
>> framework going on here. I'd also like to give my five cents:
>>
>> I was playing around with the new features of Java 8 and when it came to
>> lambdas I started to think about how useful they are when it comes to
>> testing. I always wanted to exercise how it would be to create a testing
>> framework that is tested by itself. So after visiting the XP Days this
>> year, I started to sit down and to play around with it.
>>
>> When I started I made 3 clear assumptions:
>> 1) I don't want to write any kind of assertions, at least not those that
>> come for free with Hamcrest.
>> 2) I do want to use as much of pure Java as possible, especially as
>> lambdas add a hole level of expressiveness
>> 3) I wanted to place something like a global application context to
>> configure the test system and allow plugins to register themselves for the
>> test run
>>
>>
>> While number 1 was very easy to pick up, I was wondering how to express
>> tests such that they read much nicer, but still by having pure Java code. I
>> came up with the following rules:
>> a) Use Java classes to create test classes
>> b) Use inner Java classes annotated by @Context to create hierarchical
>> contexts (like supported by the hierarchical context runner, see
>> https://github.com/bechte/junit-hierarchicalcontextrunner/wiki)
>> c) Use Java methods annotated by @Test to represent tests
>> d) All types of Tests (groups, hierarchies, methods) implement a single
>> interface, called Testable
>> e) The Testable has methods to get a name, a unique identifier and to run
>> the test returning a TestResult object
>> f) Use a TestableFactory to create the Testable objects by handing in any
>> kind of Object (a class, method, a name, etc.)
>> g) Allow plugins to register themselves with the TestableFactory to allow
>> them to participate in the process of finding a Testable
>> h) This also allows plugins to load any kind of "type" of tests, from any
>> kind of location (could be a Java class, a text-file, or what have you...)
>> i) Use a global application context to load all the configurations which
>> are provided by annotation on the test class and loaded by the framework
>> j) Instead of Rules, use Java8 lambdas to express these kind of concepts
>> (see examples below)
>> k) Use default methods on interfaces to express global/shared concepts
>> among classes / test cases
>>
>>
>> From these points, I would extract the following requirements:
>> - Test names: Within Java, they should conform to the Java Naming
>> Conventions. Outside, it could be anything the according plugin parser can
>> evaluate. :-)
>> - Runners: Runners should not be limited to one per class. Indeed, the
>> TestableFactory would allow to have multiple runners for a test, such that
>> you can have a class that runs its methods as tests and resolves additional
>> tests from a test-file or so. Think of it as possibility to have your setup
>> and teardown in Java methods while the tests cases are specified at another
>> file.
>> - Reporting: Runners should notify Listeners during the test run and
>> return a TestResult to the framework, which should gather all results and
>> report them to Reports at the end of the run. These two concept allow
>> mostly every kind of reporting that might be necessary, I think. We should
>> get rid of these kind of Description thing and there should be no need to
>> create a "tree structure" of the tests and their children right before the
>> start. I think the runners should return their results and it may be
>> combined into a tree like structure in order to represent hierarchies of
>> tests.
>> - Moreover, the TestResult class should be as abstract as possible. When
>> talking about "Jump to Source" we should think of it like an URL. The
>> source of a Testable could be almost everything, like a class, a method, a
>> txt-file or cucumber test, or what have you. Therefore, Jump to Source
>> should point to a unique name, an unique identifier that must be resolved
>> by the IDE, that wants to link back to the code / file / etc. Therefore,
>> use the default naming conventions and schemas, which should reflect almost
>> everything which is needed.
>> - Rules: There is nothing like a Rule needed. Global startup /
>> initialization / teardown can be performed by the hook within the
>> application context, once for a suite / test class / whatever. Rule for
>> handling exceptions, run in parallel, run multiple times, or to test timing
>> behavior can be easily represented by the lambdas mentioned in j). Here are
>> two examples of how this might read (you can find it at
>> https://github.com/bechte/JUT/tree/master/src/main/java/de/bechte/jut/matchers
>> )
>>
>> Example 1: Exception Handling
>> ========================
>>
>> @Test
>> public void someExceptionTest throws Exception {
>>   expectException(MyException.class).withMessage("Some error
>> message!").in(() -> {
>>     // here goes your test code without exception handling
>>     service.call("whatever");
>>   });
>> }
>>
>> Example 2: Run multiple times (parallel):
>> ===============================
>>
>> @Test
>> public void multipleRunTest throws Exception {
>>   runMultipleTimes(10).inParallel().withTimeout(1,
>> TimeUnit.SECONDS).run(() -> {
>>     // here goes your long running test code
>>     service.call("whatever");
>>   });
>> }
>>
>> - Filtering and Sorting: Filtering and Sorting should also be configured
>> during startup / bootstrapping the tests. The filters and sorters should be
>> registered within the application context and should be used by the
>> TestableFactory when searching / resolving the Testable objects. This is
>> the perfect place to perform both, filtering & sorting.
>>
>>
>> I've put together all that I have in my little prototype and I really
>> think it is worth to take a closer look at it:
>> https://github.com/bechte/JUT. Please don't take it the wrong way, this
>> is just my ideas expressed by code, as I think it is much easier to
>> understand some of the parts by looking at the code.
>>
>> I hope some of the ideas make sense and will help to create a great new
>> version for Testing in Java. :-)
>>
>>
>> Btw. I really think that Java 8 comes with so many new features and that
>> the programming model has significally changed, such that it would be a
>> great idea to "rethink" the former ideas of what Unit Testing was all about
>> and I wouldn't be to sad if we start all over and do not support older
>> versions with whatever will be the "new" framework. The changes in the Java
>> language really allow to break downward compatibility when creating
>> something much better, I think.
>>
>> All the best,
>>
>> Stefan
>>
>>
>> [Non-text portions of this message have been removed]
>>
>>
>>
>> ------------------------------------
>>
>> ------------------------------------
>>
>>
>> ------------------------------------
>>
>> Yahoo Groups Links
>>
>>
>>
>>
>