Re: Quo Vadis JUnit

I think there's likely much more to talk about than could fit in a single
thread.  Let's pick up the discussion here:

https://groups.google.com/forum/#!forum/testing-java-8

Anyone is welcome; within the group, we can start individual threads on
individual subtopics, leaving this main list open for more incremental
discussions about JUnit 4, and community troubleshooting.

   David

On Thu, Oct 23, 2014 at 5:12 PM, Stefan Bechtold [email protected]
[junit] <[email protected]> wrote:

> There are a lot of opinions, and great ideas for a new flexible testing
> framework going on here. I'd also like to give my five cents:
>
> I was playing around with the new features of Java 8 and when it came to
> lambdas I started to think about how useful they are when it comes to
> testing. I always wanted to exercise how it would be to create a testing
> framework that is tested by itself. So after visiting the XP Days this
> year, I started to sit down and to play around with it.
>
> When I started I made 3 clear assumptions:
> 1) I don't want to write any kind of assertions, at least not those that
> come for free with Hamcrest.
> 2) I do want to use as much of pure Java as possible, especially as
> lambdas add a hole level of expressiveness
> 3) I wanted to place something like a global application context to
> configure the test system and allow plugins to register themselves for the
> test run
>
>
> While number 1 was very easy to pick up, I was wondering how to express
> tests such that they read much nicer, but still by having pure Java code. I
> came up with the following rules:
> a) Use Java classes to create test classes
> b) Use inner Java classes annotated by @Context to create hierarchical
> contexts (like supported by the hierarchical context runner, see
> https://github.com/bechte/junit-hierarchicalcontextrunner/wiki)
> c) Use Java methods annotated by @Test to represent tests
> d) All types of Tests (groups, hierarchies, methods) implement a single
> interface, called Testable
> e) The Testable has methods to get a name, a unique identifier and to run
> the test returning a TestResult object
> f) Use a TestableFactory to create the Testable objects by handing in any
> kind of Object (a class, method, a name, etc.)
> g) Allow plugins to register themselves with the TestableFactory to allow
> them to participate in the process of finding a Testable
> h) This also allows plugins to load any kind of "type" of tests, from any
> kind of location (could be a Java class, a text-file, or what have you...)
> i) Use a global application context to load all the configurations which
> are provided by annotation on the test class and loaded by the framework
> j) Instead of Rules, use Java8 lambdas to express these kind of concepts
> (see examples below)
> k) Use default methods on interfaces to express global/shared concepts
> among classes / test cases
>
>
> From these points, I would extract the following requirements:
> - Test names: Within Java, they should conform to the Java Naming
> Conventions. Outside, it could be anything the according plugin parser can
> evaluate. :-)
> - Runners: Runners should not be limited to one per class. Indeed, the
> TestableFactory would allow to have multiple runners for a test, such that
> you can have a class that runs its methods as tests and resolves additional
> tests from a test-file or so. Think of it as possibility to have your setup
> and teardown in Java methods while the tests cases are specified at another
> file.
> - Reporting: Runners should notify Listeners during the test run and
> return a TestResult to the framework, which should gather all results and
> report them to Reports at the end of the run. These two concept allow
> mostly every kind of reporting that might be necessary, I think. We should
> get rid of these kind of Description thing and there should be no need to
> create a "tree structure" of the tests and their children right before the
> start. I think the runners should return their results and it may be
> combined into a tree like structure in order to represent hierarchies of
> tests.
> - Moreover, the TestResult class should be as abstract as possible. When
> talking about "Jump to Source" we should think of it like an URL. The
> source of a Testable could be almost everything, like a class, a method, a
> txt-file or cucumber test, or what have you. Therefore, Jump to Source
> should point to a unique name, an unique identifier that must be resolved
> by the IDE, that wants to link back to the code / file / etc. Therefore,
> use the default naming conventions and schemas, which should reflect almost
> everything which is needed.
> - Rules: There is nothing like a Rule needed. Global startup /
> initialization / teardown can be performed by the hook within the
> application context, once for a suite / test class / whatever. Rule for
> handling exceptions, run in parallel, run multiple times, or to test timing
> behavior can be easily represented by the lambdas mentioned in j). Here are
> two examples of how this might read (you can find it at
> https://github.com/bechte/JUT/tree/master/src/main/java/de/bechte/jut/matchers
> )
>
> Example 1: Exception Handling
> ========================
>
> @Test
> public void someExceptionTest throws Exception {
>   expectException(MyException.class).withMessage("Some error
> message!").in(() -> {
>     // here goes your test code without exception handling
>     service.call("whatever");
>   });
> }
>
> Example 2: Run multiple times (parallel):
> ===============================
>
> @Test
> public void multipleRunTest throws Exception {
>   runMultipleTimes(10).inParallel().withTimeout(1,
> TimeUnit.SECONDS).run(() -> {
>     // here goes your long running test code
>     service.call("whatever");
>   });
> }
>
> - Filtering and Sorting: Filtering and Sorting should also be configured
> during startup / bootstrapping the tests. The filters and sorters should be
> registered within the application context and should be used by the
> TestableFactory when searching / resolving the Testable objects. This is
> the perfect place to perform both, filtering & sorting.
>
>
> I've put together all that I have in my little prototype and I really
> think it is worth to take a closer look at it:
> https://github.com/bechte/JUT. Please don't take it the wrong way, this
> is just my ideas expressed by code, as I think it is much easier to
> understand some of the parts by looking at the code.
>
> I hope some of the ideas make sense and will help to create a great new
> version for Testing in Java. :-)
>
>
> Btw. I really think that Java 8 comes with so many new features and that
> the programming model has significally changed, such that it would be a
> great idea to "rethink" the former ideas of what Unit Testing was all about
> and I wouldn't be to sad if we start all over and do not support older
> versions with whatever will be the "new" framework. The changes in the Java
> language really allow to break downward compatibility when creating
> something much better, I think.
>
> All the best,
>
> Stefan
>
>
> [Non-text portions of this message have been removed]
>
>
>
> ------------------------------------
>
> ------------------------------------
>
>
> ------------------------------------
>
> Yahoo Groups Links
>
>
>
>