Using Promises and Tape for Easy Testing

The goal of writing tests for your software is maintainability over time. As you make changes to code you will want to know whether those changes have broken or otherwise altered any existing functionality. If many people are making changes to the same codebase testing becomes even more crucial.

The downside is that writing tests is tedious and often difficult! On the one hand, testing tools are often overly complicated, hard to read and extend. On the other hand, much of the code you will be writing in Node will be asynchronous, and managing concurrency doesn't get any easier when writing tests.

Enter TAP, tape and Promises.

The TAP prototocol

The Test Anything Protocol(TAP) was first implemented as a testing harness for Perl. Sample TAP output:

1..4
ok 1 - Input file opened
not ok 2 - First line of the input valid
ok 3 - Read the rest of the file
not ok 4 - Unexpected EOF

The goal of TAP is simplicity, generating textual output that is easy to produce and easy to consume. This makes it easy to pipe the output of a TAP producer to a TAP consumer. This pattern of streaming text through cooperating processes is fundamental to Unix and, by extension, to Node.js.

When using TAP with Node, unlike with more ornate testing frameworks like Mocha, there is little ceremony around setting up a test runner, an assertion library, shared global state to be set up and torn down, and so forth.

We're going to use a TAP producer for Node, a module by James Halliday named tape.

TAP output with tape

Testing with tape is easy. Consider:

var test = require('tape');

test('timing test', function (t) {

    t.plan(2);

    t.equal(typeof Date.now, 'function', 'Date.now is a function');
    var start = Date.now();

    setTimeout(function () {
        t.equal(Date.now() - start, 100, 'timeout is exactly 100 ms');
    }, 100);
});

This produces the following output:

TAP version 13
# timing test
ok 1 Date.now is a function
not ok 2 timeout is exactly 100 ms
  ---
    operator: equal
    expected: 100
    actual:   104
    at: Timeout._onTimeout ([...]/timing.js:10:11)
  ...

1..2
# tests 2
# pass  1
# fail  1

Need any help understanding what that TAP report is saying? How about understanding what the tape code is testing, or how to set up a test? Good!

Keeping it simple, tape assertions are just reflections of the native assert module. There are some added test assertions and convenience methods for grouping tests and so on, but primarily you are writing simple native Node assertions with the addition of useful output messages.

You are writing simple isEqual, isNotEqual, pass, fail, and other assertions. This is easy to write and easy to read and in practice there is rarely a need for anything more complicated. You are describing what you want to test, what you expect its value to be, and whether or not it ends up with that value.

One final bonus is that tape can just as easily be used in the browser.

Reporters

It's often useful to stream tape output to a special reporter. Typically, the goal is to provide a "prettier" formatting. The following screenshot demonstrates the output of one such reporter, tap-spec:

You can find a list of common reporters here. Adding reporters for tape is dead simple:

var test = require('tape');
var tapSpec = require('tap-spec');

test.createStream()
  .pipe(tapSpec())
  .pipe(process.stdout);

Rather than piping to stdout you might pipe to some build reporter when deploying, to a file, to the next stage of your build step, and so on. Very natural, native Node stuff, power and flexibility without boilerplate or ceremony. Streams FTW!

Take a break from reading this and go write some tests. Easy right? No complicated installs or new concepts. Just write some assertions in Plain JavaScript™ and it just works.

When you've burned tape reporting into your bones, read on about how to make testing even simpler with Promises.

Promises

Promises help you manage concurrency. If you aren't familiar with Promises you can read this introduction. For our purposes the key takeaways are:

  1. Promises enhance the readability of the often complicated control flows found in an event-driven, asynchronous programming environment like Node.js
  2. Tests are by definition error prone, and Promises make it easier to catch and handle errors.

(We will be using the bluebird Promise library for our examples. There are other implementations, including a native JavaScript implementation.)

When writing tests you will often perform asynchronous operations -- testing database calls, a filesystem read, calling some remote API, and other I/O operations. Promises can help with keeping tests readable in the face of concurrency. For example when testing a login you might want to test if you can verify user credentials and update a log somewhere. With callbacks you might write something resembling this pseudocode:

var someDBHandle;
var someFakeUserData;

someDBHandle.find(someFakeUserData, function(err, user) {
    if(err) // handle error
    someDBHandle.set('testLog', user, function(err, ok) {
        if(err) // handle error
        // send back response
    }
})

Note the callbacks and multiple error handlers. Using Promises you would write something more like this:

var Promise = require('bluebird');

somePromisifiedDBHandle.find(someFakeUserData)
.then(function(user) {
    return somePromisifiedDBHandle('testLog', user) 
})
.then(function() {
    // send back response
})
.catch(function(err) {
    // do something if errors
})

The then construct keeps your code tree flatter (nesting can go very deep very easily, often called "callback hell"). This helps with readability. However, the main advantage of Promises here is the catch construct, which allows you to localize failure across an entire testing block. Instead of writing complicated logic to manage scattered error events you handle errors in one place. You can throw anywhere in your test suite (not uncommon) and know that error will be caught in a way that you can easily handle, with a nice stack trace.

Promises are just that: an expectation of some value becoming available in the future. You are promised a notification when that value becomes available (then) or if that result will never be available (catch).

There's more. Let's go back to our tape test. Notice the t.plan(2) line? tape tests use this information to detemrine when to stop. But nobody likes to plan everything out beforehand, amirite? Wouldn't it be nice to just write as many or as few tests as you'd like (even change that number dynamically) and have everything just work?

Enter blue-tape. This is a lightweight wrapper around tape that accomplishes the above goal. Combining blue-tape with Promises you can now write your tests like this:

var test = require('blue-tape');

test('blue test', function (t) {

    return somePromisedValue()
    .then(function(value) {
        t.isEqual(value, 1, 'Got 1 as a value')
    })
    .then(someOtherTest())
    // run as many tests in here as you'd like
    .then(...)
    .then(...)  
});

Notice: no plan necessary!

This allows you to require independent test suites, dynamically include/exclude sub-tests based on previous results, and so forth. All you need to do is write your tests within Promsies (which you were going to do anyway, right?). Very powerful stuff.

Let's wrap all these concepts up into a simple module that makes testing dead simple: surveyor.

surveyor

The surveyor module lets you modularize your tests into individual files that will all be run in parallel. You simple create a test folder with a spec subfolder containing your test files:


test
└─┬ spec
  └── test1.js
  └── test2.js
└──index.js

The index.js file is a simple configuration file:

require('surveyor')({
    testDir: 'pathToTestFolder', // eg. __dirname
    specDir: 'specFolder', // Optional. Default is 'spec', eg. {testDir}/spec
    globalFixtures: [ // Optional. These files are expected to return test scopes
        'globals/A',
        'globals/B'
    ]
});

We're going to ignore the globalFixtures as that goes beyond the scope of this article. These simply provide contexts that all test files will be executed within. For example, if you needed an express server to be available to all tests you would add it here.

The key files are the ones in your spec folder -- your test files. Here is what one of those might look like:

module.exports = function(test, Promise) {
    return getRecord()
    .then(function(rec) {
        test.ok(someTestAgainstRec);
        return updateUser()
    })
    .then(function(user) {
        test.ok(someTestAgainstUser);
        return ...
    });
};

Running tests is equally simple:

$ node test

You will get a pretty result for each of your test files, something like:

./test/spec/test1.js
✔ Something passed
./test/spec/test2.js
✔ Something passed
✖ Something failed
✔ Something else passed
4 test(s) planned
3 test(s) passed
1 test(s) failed

As long as your test files use Promises throughout there is no additional setup or configuration to do. Add or remove tests from the files and your suite will continue to work -- no plan-ing or other modifications necessary! Additionally, reporting is built right in so there is no need to do any other work to get your tests up and running.

Any number of test files can be written in this way and placed in your spec directory. These will be run in parallel, which leads to both fast execution and a nicely organized, modular, testing system. As well, it should be easy to read the tests and understand exactly what is being tested. If your tests are comprehensive, this means reading the tests will be like reading documentation on each of the major parts of your application.

Summary

Hopefully this short introduction has demonstrated to you how using tape and Promises can make writing tests simple, both in the writing and in the reading. Using the TAP protocol lets you build very tight testing pipelines right into any typical Node build system. blue-tape brings Promises to tape in a way that tames concurrency and lets you focus on writing tests, not structuring your testing boilerplate. Finally, the surveyor module is a one-stop-shop for setting up testing -- just drop some test files into one folder and run node test. It couldn't be simpler.

Share This