JavaScript Generators and Preventing Callback Hell

Node.js is the one of the best technologies for building I/O intensive web applications. JavaScript’s single threaded nature provides many advantages over the thread based execution model found in other programming languages for this class of applications. But, these advantages come at a price. Writing large Node applications can be difficult due to callback hell and asynchronous error handling.

There are a number of solutions for overcoming these issues. Fibers and Promises are two of them. Some developers prefer using these solutions, but it all comes down to personal preference. The yet to be released ECMAScript version 6 also introduces generators as a solution to callback hell. This articles gives you a proper introduction to generators and shows how they can be used to solve the issues mentioned above.

Prerequisites

As I mentioned, generators are a new feature in JavaScript. The current stable release of Node (0.10.x) does not include generators. So, we’ll need to install the current unstable 0.11.x release in order to use generators. Once you’ve installed 0.11.x, you can enable generators by passing the --harmony-generators flag to Node, as shown below.

node --harmony-generators <filename.js>

Generators 101

Simply put, generators are a type of function (note the * in the following code sample) which act as an iterator. Generators can contain any valid JavaScript code. Let’s write our first generator (shown below).

function* HelloGen() {
  yield 100;
  yield 400;
}

var gen = HelloGen();

console.log(gen.next()); // {value: 100, done: false}
console.log(gen.next()); // {value: 400, done: false}
console.log(gen.next()); // {value: undefined, done: true}

yield is a special keyword which emits a new item from the generator. We can use next() to get values from a generator. Once we reach the end of the iterator, the returned object will contain done: true. Any data type can be yielded, including functions, numbers, arrays, and objects.

Values can also be passed to generators as shown below.

function* HelloGen2() {
  var a = yield 100;
  var b = yield a + 100;

  console.log(b);
}

var gen2 = HelloGen2();

console.log(gen2.next());     // {value: 100, done: false}
console.log(gen2.next(500));  // {value: 600, done: false}
console.log(gen2.next(1000)); // {value: undefined, done: true}
// prints 1000

Preventing Callback Hell

So, how can generators be used to avoid callback hell? First, you need to understand a simple technique that we will be using heavily with generators to write code without callbacks.

Understanding Thunks

A thunk is a partially evaluated function which accepts a single callback as the argument. Within generators, we’ll be yielding thunks to write programs without callbacks. A simple thunk is shown below.

function(callback) {
  fs.readFile('myfile.md', 'utf8', callback);
}

Thunks can also be created dynamically as shown below.

function readFile(filename) {
  return function(callback) {
    fs.readFile(filename, 'utf8', callback);
  };
}

Using co

co is a nice module which helps to use thunks and generators together to create Node.js applications without callbacks. I’ll show you how it works internally later. For now let’s try co, which can be installed using the command npm install co. A simple application which uses co and the readFile() thunk from the previous example is shown below.

var co = require('co');

co(function* () {
  var file1 = yield readFile('file1.md');
  var file2 = yield readFile('file2.md');

  console.log(file1);
  console.log(file2);
})();

As you can see, we are no longer using callbacks. This gives us a simple way to write large modular Node apps easily.

How co Works Internally

You might be wondering how co works internally. Here’s how it works its magic.

• First, it calls next(null) and gets a thunk.
• Then, it evaluate the thunk and saves the result.
• Then, it calls next(savedResult).
• Repeat these steps until next() returns {done: true}.

If you prefer sample code, here’s a minimal version of co written to show you how it works internally. co is more complex than this, since it does better error handling and supports promises as well.

function co(generator) {
  var gen = generator();

  function nextItem(err, result) {
    var item = gen.next(result);

    if (!item.done) {
      item.value(nextItem);
    }
  }

  nextItem();
}

Modules That Can Be Used with co

co can be used with any module that utilizes thunks. Unfortunately, there are not many modules that currently make use of thunks. You can see the complete supported list here. With simple utilities like thu and thunkify, you can wrap any Node module as thunks for use with co.

Conclusion

Generators are fairly new, and generally unavailable. However, the Node community seems to be showing a lot of interest. One of the best example is the release of Koa. It is a generators friendly clone of Express built by the same team who built Express. I’m sure as time goes on, there will be increased support for generators from the community.