Asynchronous unit tests in FlexUnit 4

FlexUnit looks like a fairly decent Flex unit testing framework and we've started using it in our new Flex project. Unfortunately FlexUnit has pretty much zero documentation, so unless you can find a blog post from someone about how to do something you'll have to just figure it out from scratch.

My coworker and I end up having to dig into the FlexUnit code occasionally to figure out how things work. The latest instance of this for me was to try and figure out how to test asynchronous events. I found a couple posts showing that it was possible with basic examples, but no real explanation of the different options.

I've managed to get a few different types of async tests working, so I figured I'd give examples of the different async methods provided by the Async FlexUnit class.

Example Background

The examples I'm going to provide here are real tests that were written for a class called PersistErrorMonitor. The examples might be a little easier to understand with a basic understanding of the class being tested, so here you go.

The persistent error monitor uses some heuristics to detect persistent errors (in my case, network errors, but it's generic). It keeps track of all error and success transitions within a fixed sample time and looks for either of the following cases:

• More than Max Failures distincs failures within the sample time
• More than Max Failure Time total time in a failure condition within the sample time

So, if it is constructed with a max failure count of 5, failure time of 30 seconds, and sample time of 60 seconds then if will signal that it found a persistent error if it sees 6 distinct failures (transition from success to failure) or has seen 30 seconds or more in a failure state within the last 60 seconds.

Async Testig Basics

Creating an asynchronous test involves the following steps:

1. Declare the test as asynchronous and give it a timeout
2. Use the org.flexunit.async.Async class methods to create the event handlers for any asynchronous events you need to handle.

There are a few different helpers that org.flexunit.async.Async provides:

• proceedOnEvent - Tells FlexUnit to wait for the given event, failing if it doesn't occur within the provided timeout.
• failOnEvent - Tells FlexUnit to fail the test if the given event is raised within the timeout.
• handleEvent - Tells FlexUnit to handle an event asynchronously as part of the test. This lets you provide your own event handler that will be called.
• asyncHandler - Creates and returns an event handler that can be added to an event dispatcher.
• asyncResponder - Not sure on this one, I haven't used it.

In the following sections I'll give an example test that uses one of those methods.

Async.proceedOnEvent

Async.proceedOnEvent should be used if your test expects a particular event to be raised and doesn't need to test anything other than that the event was raised.

The signature for proceedOnEvent is:

public static function proceedOnEvent( testCase:Object, target:IEventDispatcher,
    eventName:String, timeout:int=500, timeoutHandler:Function = null ):void

The usage is pretty straightforward. Set up your test to proceed on the event, then do whatever you need that is supposed to trigger the event. Here's an example where I'm testing to make sure the persistent error event is raised when the maximum failure time is reached:

[Test(async, timeout=1000)]
public function testMaxFailureTimeDetection():void {
  var monitor:PersistErrorMonitor = new PersistErrorMonitor(5, 200, 500,
      100);

  Async.proceedOnEvent(this, monitor, PersistErrorEvent.ERROR_DETECTED,
      500);

  monitor.addStatus(false);
}

Make sure the timeout you define for the test in the [Test] metadata tag is long enough that there is time for your event to fire if it works, but short enough that if the event fails to fire your tests don't block for too long.

proceedOnEvent also takes a timeout argument. I generally use that as the "real" timeout for the event, and just set the test's overall timeout to be a bit larger than that.

If the event is raised then the test will "proceed", and since there are no more asserts it will pass. If the event fails to be raised then the timeout will be hit and the test will fail.

Async.failOnEvent

This is the exact opposite of proceedOnEvent. Use it if you want to make sure that an event is not raised in a certain case. You give failOnEvent a timeout and as long as the event is not raised within that time the test will pass. The signature is:

public static function failOnEvent( testCase:Object, target:IEventDispatcher,
    eventName:String, timeout:int=500, timeoutHandler:Function = null ):void

Note that failOnEvent doesn't actually do anything with the timeoutHandler argument, so don't expect anything passed there to be called (like I did). Hopefully they'll either remove the argument or actually use it in the future.

Here's an example that tests to make sure the persistent error event is not raised when it shouldn't be:

[Test(async, timeout=1000)]
public function testNoEventWhenNoError():void {
  var monitor:PersistErrorMonitor = new PersistErrorMonitor(5, 200, 500,
      100);

  Async.failOnEvent(this, monitor, PersistErrorEvent.ERROR_DETECTED, 500);

  monitor.addStatus(false);
  monitor.addStatus(true);
}

Try not to use timeouts that are too large for failOnEvent, as it will block the tests until that timeout has completed (unless an event was raised causing the test to fail).

Async.handleEvent

You'd use handleEvent when you need to test things after an event has been raised. If the event is not raised within the timeout time it will count as a failure, so using handleEvent does imply that the event is always expected. Here's the signature:

public static function handleEvent( testCase:Object, target:IEventDispatcher,
    eventName:String, eventHandler:Function, timeout:int=500, passThroughData:Object = null,
    timeoutHandler:Function = null ):void

Here's an example where we want to check to make sure the event's data is correctly populated:

[Test(async, timeout=1000)]
public function testEventContainsFailureInfo():void {
  var monitor:PersistErrorMonitor = new PersistErrorMonitor(5, 200, 500,
      100);

  var check:Function = function(e:PersistErrorEvent, ... args):void {
    Assert.assertEquals("Fail count", 2, e.failureCount);
    Assert.assertTrue("Fail time", e.failureTime >= 200);
  }
  Async.handleEvent(this, monitor, PersistErrorEvent.ERROR_DETECTED,
      check, 500);

  monitor.addStatus(false);
  monitor.addStatus(true);
  monitor.addStatus(false);
}

One thing to watch out for is that the event handler is passed 2 arguments, not just the event. I'm not sure what that second argument is, but your method should take it to make sure you don't get an argument mismatch error at runtime. Generally I just use the ... vararg declaration.

Async.asyncHandler

This will return an event handler you can add to an event dispatcher manually. Generally you should probably just use handleEvent as it does the extra work for you, but it might be useful in some cases. Here's the signature:

public static function asyncHandler( testCase:Object, eventHandler:Function,
    timeout:int, passThroughData:Object = null, timeoutHandler:Function = null ):Function

Here's the example from Async.handleEvent rewritten to use Async.asyncHandler:

[Test(async, timeout=1000)]
public function testEventContainsFailureInfo():void {
  var monitor:PersistErrorMonitor = new PersistErrorMonitor(5, 200, 500,
      100);

  var check:Function = function(e:PersistErrorEvent, ... args):void {
    Assert.assertEquals("Fail count", 2, e.failureCount);
    Assert.assertTrue("Fail time", e.failureTime >= 200);
  }

  var handler:Function = Async.asyncHandler(this, check, 500);
  monitor.addEventListener(PersistErrorEvent.ERROR_DETECTED, handler); 

  monitor.addStatus(false);
  monitor.addStatus(true);
  monitor.addStatus(false);
}

Just like handleEvent the event handler function you provide needs to take 2 arguments, not just the event.

Time-based tests

Sometimes you may need to write a test that doesn't expect a particular event to occur but instead needs to wait some time and then either run more logic to trigger the test case or just test things directly. To do this with the FlexUnit async support you'd need to kick off a Timer and use Async.handleEvent to delay your code.

This kind of thing could be kind of common, so here's a helper method you can use to make it easier:

public function runAfterTimeout(testClass:Object, time:Number,
    toRun:Function):void
{
  var runWrapper:Function = function(... args):void { toRun(); };
  var t:Timer = new Timer(time);
  Async.handleEvent(testClass, t, TimerEvent.TIMER, runWrapper, time + 200);
  t.start();
}

With that declared you can delay parts of your test easily, either passing closures in or other explicit methods you've defined. I'm just using closures as it keeps all the test code contained in a single method in the unit test class, but that's just a style decision. Here's an example where I use two calls to runAfterTimeout to set up a situation where a failure state has partially left the sample time and want to make sure it's properly trimming the time:

[Test(async, timeout=2000)]
public function testFailureTimeIsTrimmedToSampleTime():void {
  var monitor:PersistErrorMonitor = new PersistErrorMonitor(5, 1000, 1000,
      100);

  monitor.addStatus(false);

  runAfterTimeout(this, 800, function():void {
    monitor.addStatus(true);
  });

  runAfterTimeout(this, 1500, function():void {
    assertClose("Fail time", 300, monitor.failureTime, 100);
    Assert.assertEquals("Fail count", 1, monitor.failureCount);
  });
}

Be careful that the timeout you declare in your test metadata for the function is long enough for all your runAfter blocks to complete. A couple times I tweaked the timing of the runAfter blocks and forgot to update the test's timeout, causing errors I didn't expect.

Also, since I reference it there, here's the source of the assertClose method I used:

public function assertClose(message:String, expected:Number, actual:Number,
    range:Number):void
{
  if( ! (actual > expected - range) && (actual < expected + range)) {
    Assert.fail(message + "Got " + actual + " expected " + expected + " +- " +
        range);
  }
}

---

FlexUnit is utterly lacking in documentation. So here's some on the async testing features.