Skip to content

OHHTTPStubs and asynchronous tests

Jump to bottom
AliSoftware edited this page Jul 31, 2014 · 13 revisions

Asynchronous Testing

As OHHTTPStubs is a library generally used in Unit Tests invoking network requests, those tests are generally asynchronous. This means that the network requests are generally sent asynchronously, in a separate thread or queue different from the thread used to execute the test case.

As a consequence, you will have to ensure that your Unit Tests wait for the requests to finish (and have their response arrived) before performing the assertions in your Test Cases, because otherwise your code making the assertions will execute before the request had time to get its response.

Bad example: when you don't wait for the async operation

For example, this won't work, because sendAsynchronousRequest:queue:completionHandler: will return immediately (triggering the networking request in the background) and thus the testFoo method will reach its end before the completionHandler block had time to be called.

So your test framework will see that the test didn't trigger any assertion failure and will mark the test as succeeded, even if the completionHandler block is called later and trigger a (late) assertion failure because data is nil, but it would be too late.

- (void)testFoo
{
  NSURLRequest* request = ...
  [NSURLConnection sendAsynchronousRequest:request
                                     queue:[NSOperationQueue mainQueue]
                         completionHandler:^(NSURLResponse* response, NSData* data, NSError* error)
   {
     NSAssertNotNil(data, @"Received data should not be nil");
   }
  ];
  // The rest of the code below will continue to execute without waiting for the request to have its response
}

Good example (a solution before Xcode 6's XCTestExpectation)

The correct way to write your Unit tests it thus to wait for the asynchronous request to have its response before doing any assertion on it.

Before Xcode 6, one way to do it was to make your test loop using a while loop — which would executes [[NSRunLoop currentRunLoop] runUntilDate:...] or CFRunLoopRunInMode to be sure the main RunLoop continue to run while we wait. And only breaks out of the loop if a flag is set to YES or a timeout is reached. Then in your completionHandler, simply set the aforementioned flag to YES to let the test know that it can continue and check its assertions.

- (void)testFoo
{
  NSURLRequest* request = ...
  __block BOOL responseArrived = NO;
  __block NSData* receivedData = nil;
  [NSURLConnection sendAsynchronousRequest:request
                                     queue:[NSOperationQueue mainQueue]
                         completionHandler:^(NSURLResponse* response, NSData* data, NSError* error)
   {
     receivedData = data;
     responseArrived = YES;
   }
  ];

  // Wait for the asynchronous code to finish and for completionHandler block to be called… or timeout
  NSDate* timeoutDate = [NSDate dateWithTimeIntervalSinceNow:timeout];
  while (!responseArrived && ([timeoutDate timeIntervalSinceNow]>0))
    CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0.01, YES);

  // By the time we reach this code, the while loop has exited
  // so the response has arrived or the test has timed out
  NSAssertNotNil(data, @"Received data should not be nil");
}

A nice solution with the brand new Xcode 6's XCTestExpectation

With Xcode 6, XCTest now has a new feature, called XCTestExpectation, just to do that and to handle testing in asynchronous code. Mattt even talks about it here in one of its great NSHipster's articles.

The idea behind this is to:

  • Create an XCTestExpectation using the expectationWithDescription: method, giving it a nice custom description
  • Calling the waitForExpectationsWithTimeout:handler: method when you want to wait for your asynchronous calls to end
  • In your completionHandler or whatever method your asynchronous operation calls when it finishes, call fulfill on the previously created XCTestExpectation object to mark it as… fulfilled.

Then, as its name states, waitForExpectationsWithTimeout:handler: will wait for all previously-declared expectations to be fulfilled, or for the timeout to be reached, and call the block passed to its handler: parameter (if one has been provided). It will also automatically fail if there was some expectations left unfulfilled when the timeout was reached.

- (void)testFoo
{
  NSURLRequest* request = ...
  XCTestExpectation* responseArrived = [self expectationWithDescription:@"response of async request has arrived"];
  __block NSData* receivedData = nil;
  [NSURLConnection sendAsynchronousRequest:request
                                     queue:[NSOperationQueue mainQueue]
                         completionHandler:^(NSURLResponse* response, NSData* data, NSError* error)
   {
     receivedData = data;
     [responseArrived fulfill];
   }
  ];

  [self waitForExpectationsWithTimeout:timeout handler:^{
    // By the time we reach this code, the while loop has exited
    // so the response has arrived or the test has timed out
    NSAssertNotNil(data, @"Received data should not be nil");
  }];
}

XCTestExpecation Support in Xcode5

Xcode 6 is quite new (by the time I'm writing this, it is even still in Beta4) and people will probably keep using Xcode 5 for a while during the transition. And the XCTestExpectation feature is only available starting Xcode 6… so what about people still using Xcode 5?

Well hopefully for you, I re-implemented XCTestExpectation myself for those people still using Xcode 5!

  • It only gets compiled when used in Xcode 5 (namely __IPHONE_VERSION_MAX_ALLOWED < 80000 or __MACOSX_VERSION_MAX_ALLOWED < 101000); when using Xcode 6, the native, Apple's implementation is used instead
  • My implementation uses the exact same API than Xcode 6's XCTestExpectation, so you won't have to change a single line of code in your unit tests when you will switch from Xcode 5 to Xcode 6.

Feel free to use it in your own Unit Tests if you want to take advantage of this nice XCTestExpectation API early on (and to be ready to migrate to Xcode 6 transparently when ready)! (and drop me a line to let me know if it was useful to you ;))

Common errors when not handling async tests properly

To end this article, one common error you may see if you [OHHTTPStubs removeAllStubs] at the end of your Unit Tests (for example in the tearDown method), but failed to wait for your asynchronous test to have a response before letting the test case end. In that case, you may encounter the issue described here. Be aware that when having this error, it probably means that you forgot to wait for your asynchronous operations to finish!

Clone this wiki locally