Asynchronous APIs Using the Fetch API and ES6 Generators

October 5, 2015

ECMAScript 6 (a.k.a. ECMAScript 2015 or ES6) brings a number of new features to JavaScript which will make the language a good fit for large applications. One of these features is better support for asynchronous programming using promises and generators. Another is the addition of the Fetch API which aims to replace XMLHttpRequest as the foundation of communication with remote resources.

The Fetch API’s methods return ES6 Promise objects, which can be used in conjunction with generators to form the basis of complex asynchronous operations. This could be anything from a chain of asynchronous operations, where each operation depends on the value returned by the previous one, to an asynchronous call that has to be made repeatedly to a server to get the latest update.

In this article we will see how the Fetch API can be used in conjunction with generators to build asynchronous APIs. The Fetch API is currently supported in Chrome, Opera, Firefox and Android browsers. We have a polyfill available from GitHub for unsupported browsers.

As ever, the code for this article can be found on our GitHub repository and there is a demo of the final technique at the bottom of the article.

Generators for Asynchronous Operations

Tip: If you need a refresher on what generators are and how they work, check out: ECMAScript 2015: Generators and Iterators

So how can we use generators to perform async operations? Well, if we analyze the way generators work we will find the answer.

A generator function implementing an iterator has the following structure:

function *myIterator(){
  while(condition){
    //calculate next value to return
    yield value;
  }
}

The yield keyword is responsible for returning a result and halting execution of the iterator function until it is next invoked. It also keeps the state of the function instead of rerunning everything when next you call it, effectively remembering the last place it left off.

We can re-imagine the above function without while loop as follows:

function *myIterator(){
  //calculate value 1
  yield value1;

  //calculate value 2
  yield value2;
  ...

  //calculate value n
  yield valuen;
}

The behavior of the function will be identical in both of the above cases. The only reason for using the yield keyword is to pause the execution of the function until the next iteration (which in itself seems kind of asynchronous). And as the yield statement can return any value, we can also return promises and make the function run multiple asynchronous calls.

Using Generators with the Fetch API

Tip: For a refresher on the Fetch API, check out: Introduction to the Fetch API

As mentioned earlier the Fetch API is intended to replace XMLHttpRequest. This new API provides control over every part of an HTTP request and returns a promise that either resolves or rejects based on the response from the server.

Long Polling

One of the use cases where the Fetch API and generators can be used together is long polling. Long polling is a technique in which a client keeps sending requests to a server until it gets a response. Generators can be used in such a case to keep yielding the responses until the response contains data.

To mimic long polling, I included an Express REST API in the sample code that responds with weather information of a city after five attempts. The following is the REST API:

var polls=0;

app.get('/api/currentWeather', function(request, response){
  console.log(polls, polls<5);
  if(polls < 5){
    console.log("sending...empty");
    polls++;
    response.send({});
  }
  else{
    console.log("sending...object");
    response.send({
      temperature: 25,
      sky: "Partly cloudy",
      humid: true
    });
    polls = 0;
  } 
});

Now, let’s write a generator function that calls this API multiple times and returns a promise on every iteration. Being on the client side, we don’t know after how many iterations we will get data from the server. So, this method will have an infinite loop pinging the server on every iteration and returning the promise on every occasion. Following is the implementation of this method:

function *pollForWeatherInfo(){
  while(true){
    yield fetch('/api/currentWeather',{
      method: 'get'
    }).then(function(d){
      var json = d.json();
      return json;
    });
  }
}

We need a function to keep calling this function and checking if the value exists after the promise resolves. It will be a recursive function that invokes the next iteration of the generator and only stops the process when it finds a value returned from the generator. The following snippet shows the implementation of this method and a statement that calls this method:

function runPolling(generator){
  if(!generator){
    generator = pollForWeatherInfo();
  }

  var p = generator.next();
  p.value.then(function(d){
    if(!d.temperature){
      runPolling(generator);
    } else {
      console.log(d);
    }
  });
}

runPolling();

As we see here, the first call to the function runPolling creates the generator object. The next method returns an object with a value property which in our case contains a promise returned by the fetch method. When this promise resolves, it will either contain an empty object (returned if the polls variable is below 5), or an object containing the desired information.

Next, we check for the temperature property of this object (which would indicate success). If it’s not present we pass the generator object back to the next function call (so as not to lose the state of the generator) or we print the value of the object to the console.

To see this in action, grab the code from our repo, install the dependencies, start the server, then navigate to http://localhost:8000. You should see the following results in the shell:

0 true
sending...empty
1 true
sending...empty
2 true
sending...empty
3 true
sending...empty
4 true
sending...empty
5 false
sending...object

And the object itself logged to the browser console.

Multiple Dependent Asynchronous Calls

Quite often, we need to implement multiple dependent asynchronous calls, where each successive asynchronous operation depends on the value returned by the the preceding asynchronous operation. If we have a group of such operations and they have to be called multiple times, we can put them together in a generator function and execute it whenever we need it.

To demonstrate this, I will be using GitHub’s API. This API provides us access to basic information on users, organizations and repos. We will use this API to get the list of contributors to a random repo of an organization and display the fetched data on the screen.

For this we need to make calls to three different endpoints. These are the tasks to be performed:

Get details of the organization
If the organization exists, get the organization’s repos
Get contributors to one of the organization’s repos (selected at random)

Let’s create a wrapper function around Fetch API to avoid repeating the code to create the headers and build the request object.

function wrapperOnFetch(url){
  var headers = new Headers();
  headers.append('Accept', 'application/vnd.github.v3+json');
  var request = new Request(url, {headers: headers});

  return fetch(request).then(function(res){
    return res.json();
  });
}

The following function consumes the above function and yields a promise for each invocation:

function* gitHubDetails(orgName) {
  var baseUrl = "https://api.github.com/orgs/";
  var url = baseUrl + orgName;

  var reposUrl = yield wrapperOnFetch(url);
  var repoFullName = yield wrapperOnFetch(reposUrl);
  yield wrapperOnFetch(`https://api.github.com/repos/${repoFullName}/contributors`);
}

Now, let’s write a piece of logic to call the above function to get the generator and then use the values obtained from the server to populate the UI. As every call to the generator’s next method returns a promise, we will have to chain these promises. The following is the skeleton of the code using the generator returned by the above function:

var generator = gitHubDetails("aspnet");

generator.next().value.then(function (userData) {
  //Update UI

  return generator.next(userData.repos_url).value.then(function (reposData) {
    return reposData;
  });
}).then(function (reposData) {
  //Update UI

  return generator.next(reposData[randomIndex].full_name).value.then(function (selectedRepoCommits) {
    //Update UI
 });
});

To see this in action, as detailed above, grab the code from our repo, install the dependencies, start the server, then navigate to http://localhost:8000. Or just check out the demo below (try rerunning it).

Demo

See the Pen Multiple Dependent Asynchronous Calls With the Fetch API by SitePoint (@SitePoint) on CodePen.

Conclusion

In this article I have demonstrated how the Fetch API can be used in conjunction with generators to build asynchronous APIs. ECMAScript 6 will bring a slew of new features to the language and looking for inventive ways to combine them and harness their power can often bring outstanding results. But what do you think? Is this a technique we can start using in our apps today? I would love to hear your thoughts in the comments.

JavaScript: Novice to Ninja, 2nd Edition