-
Notifications
You must be signed in to change notification settings - Fork 6
The Red Test Module
The Red Test module provides simple interface for writing and running asynchronous tests under the JUnit framework.
First things first - what is wrong with just using JUnit as it it?
Well, when you write synchronous code, you depend on some function or method to return in order to indicate - it's operation is completed. JUnit is depending on that to mark the test as successful. If an exception was thrown the test is failed, if we want to assert something, we assert it inside the test method body, right? And If we want to wait for something we use Thread.sleep().
When writing asynchronous code, a function may return, but the operation may still be in progress. An exception may be thrown on some nested callback on a different thread without JUnit even knowing about it, we can use JUnit assert interface - but it will not fail the test since JUnit was not there to catch the assertion error, and we definitely don't want the thread to go to sleep!
All of those issues are addressed by the Red Test module, and some more:
- Declaring an asynchronous test.
- Waiting for async callbacks.
- Assertions inside callbacks.
- Schedueling delayed tasks.
- Validating time periods.
- Timeouts.
In order to write an async test you should first run the test class with RedTestRunner: @RunWith(RedTestRunner.class).
This runner is backwards compatible so that at this point everything remains the same.
Now, methods annotated with JUnit Test annotation will be executed as plain synchronous JUnit tests.
In order to transform such a method to an async test, it should receive one parameter of type RedTestContext.
A RedTestContext is the heart and soul of the async test. More about it below.
Now that we have an async test, let's define some async operations inside it. It we are to leave the test empty, it'll run like JUnit would run it by default, and finish with success.
If we are to perform some async operation, this operation will not be over when the call to it will return, right? So we need a way to indicate that the test should wait for it's computation.
First - let's consider an example async method readFromDB, which receive a callback that handles a String result. For example:
readFromDB(status -> {
if (status.success()) {
String result = status.result(); // returns the result
} else {
Throwable cause = status.cause(); // returns the failure cause
}
});Now, if we are to write a test calling this method and asserting it's result, we wan't JUnit to know that the computation is still in progress when the test method returns, but then we need some way to indicate the computation status. Enters - RedTestContext Forks.
Each async call should create a single fork of the test context. A fork is created using
RedTestContext.Fork fork = redTestContext.fork();If no fork is created until the test method returns, and no exception was thrown during that period, the test succeeds. If we fork the context, each fork should successfully complete in order for the test to complete.
Let's write a test making nested readFromDB calls.
@Test
public void test(RedTestContext redTestContext) {
RedTestContext.Fork fork1 = redTestContext.fork();
RedTestContext.Fork fork2 = redTestContext.fork();
readFromDB(status -> {
if (status.success()) {
fork1.complete();
readFromDB(status2 -> {
if (status2.success()) {
fork2.complete();
} else {
fork2.fail(status2.cause());
}
});
} else {
fork1.fail(status.cause());
}
});
}Note that all forks must be made before the test method returns. Declaring all forks at the top of the method is considered best practice.
Each fork may be completed by calling fork.complete(), and failed by calling either:
fork.fail(Throwable cause)fork.fail(String cause)-
fork.fail().
Either of those is equivalent to calling
redTestContext.fail(Throwable cause)redTestContext.fail(String cause)redTestContext.fail()
respectively.
Furthur info can be found on the API reference: https://avivcarmis.github.io/java-red/apidocs/latest/io/github/avivcarmis/javared/test/RedTestContext.html
Now that we have an async test making non blocking calls, we might want to write some assertions to be validated during test execution. JUnit introduces a wide verity of assertion methods available under JUnit Assertion. These assertion methods are built to throw an assertion error in case the assertion fails, but as we mentioned, JUnit might not be there to catch those errors and fail the test.
RedTestContext instance exposes the entire JUnit assertion interface under redTestContext.assertions, and those assertions can be executed from every thread, at any time, as long as the test is still active (some fork has not yet completed).
Let's add an assertion to out test:
@Test
public void test(RedTestContext redTestContext) {
RedTestContext.Fork fork1 = redTestContext.fork();
RedTestContext.Fork fork2 = redTestContext.fork();
readFromDB(status -> {
if (status.success()) {
redTestContext.assertions.assertNotNull(status.result());
fork1.complete();
readFromDB(status2 -> {
if (status2.success()) {
redTestContext.assertions.assertNotNull(status2.result());
fork2.complete();
} else {
fork2.fail(status2.cause());
}
});
} else {
fork1.fail(status.cause());
}
});
}If the produced result is null, the test will fail.
Furthur info can be found on the API reference: https://avivcarmis.github.io/java-red/apidocs/latest/io/github/avivcarmis/javared/test/RedTestContext.Assertions.html
A common practice when testing async execution is to schedule delayed tasks to be executed. RedTestContext can be used to help with that! A call to either
redTestContext.scheduleTask(long delay, TimeUnit unit, Runnable runnable)redTestContext.scheduleTask(long delayMillis, Runnable runnable)
may be made from anywhere within the context to schedule execution of a given runnable with given delay.
Another common practice when testing async execution is validation of time periods. Namely, validating that from a certain point of the execution to another, a certain period of time has not passed, or has passed. This may be performed using a test context TimingValidator. A TimingValidator instance is acquired through RedTestContext.timingValidator(), this marks the starting point of a validator. Then, this validator instance may be called to assert a certain period of time did or did not pass since the moment of it's creation. This is done through a TimingValidator instance's:
validatePassed(long time, TimeUnit unit)validatePassed(long timeMillis)validateNotPassed(long time, TimeUnit unit)validateNotPassed(long timeMillis)
Furthur info can be found on the API reference: https://avivcarmis.github.io/java-red/apidocs/latest/io/github/avivcarmis/javared/test/RedTestContext.TimingValidator.html
Since synchronous execution usually tends less to freeze, JUnit sets no default timeout. This means that a frozen test will not fail until the test is manually stop.
Async code can easily freeze - this may happen when we forget to pass some parameter, or forget to register some callback, or it can happen when we ignore some thrown exception. To this end, Red test overrides JUnit default timeout to 60 seconds. This can be normally altered through Test annotation timeout attribute.
Let's talk about callbacks. The Red Test executor runs the test method, waits for it to return, waits for all the forks to successfully complete, and then validates no assertion errors were thrown. When callbacks are executed, like in readFromDB example from above, the Red Test runner has no way to know about this callback or the thread executing it.
Still, it's very likely that this callback will throw an exception. In this case, nor Red Test executor or JUnit will be there to catch it. This does not mean that the test will pass, since the exception is thrown and the call to fork.complete() has not yet been performed, but it does mean that the fork will remain uncompleted, and the test will freeze until the entire test times out.
If your'e fine with a few waisted seconds, that's great. If not you may either wrap your callbacks with try, catch blocks, or altering the test timeout.