Start learning
Part 1/Generators Iterators

Async iteration and generators

Some data doesn’t exist yet when you start looping over it. It shows up later: a network response, a chunk of a file, the next page of an API. Regular iteration can’t wait. Asynchronous iteration can — it lets you pull values one at a time, pausing between each until the value is ready. Async generators make writing those producers short and readable.

We’ll build up slowly. First a quick refresher on ordinary iterables, then the async version, then generators, then async generators, and finally a real GitHub pagination example you could drop into a project.

A refresher on iterables

Say you have a plain object like floors:

let floors = {
  bottom: 2,
  top: 6
};

You want for..of to walk it from 2 to 6. Out of the box that won’t work, because a plain object has no idea what “next value” means. You have to teach it. That teaching is done through a method keyed by the well-known symbol Symbol.iterator.

Here is the contract, and it’s worth holding in your head because the async version mirrors it exactly:

  • for..of calls the object’s Symbol.iterator method once, at the start. That method returns an iterator — an object with a next() method.
  • Every turn of the loop calls next().
  • next() returns { done: false, value: ... } while there’s more to give, and { done: true } when it’s finished.
for..of
the loop
→
Symbol.iterator()
called once → returns iterator
→
next()
called each turn →
{ value, done }
How for..of drives an iterator: one Symbol.iterator call, then repeated next() calls until done is true.

Here’s floors wired up:

let floors = {
  bottom: 2,
  top: 6,

  [Symbol.iterator]() { // called once, at the start of for..of
    return {
      current: this.bottom,
      last: this.top,

      next() { // called every iteration to get the next value
        if (this.current <= this.last) {
          return { done: false, value: this.current++ };
        } else {
          return { done: true };
        }
      }
    };
  }
};

for (let value of floors) {
  alert(value); // 2, then 3, then 4, then 5, then 6
}

Notice the split: Symbol.iterator returns a separate object that holds the loop state (current, last). Keeping state off floors itself means two loops over the same floors don’t step on each other. If any of this feels shaky, the Iterables chapter covers it in full.

Async iterables

Everything above assumes values are ready the instant next() is called. That breaks the moment a value depends on a timer, a disk read, or a server. You can’t return { done: false, value } when you don’t have the value yet — you only have a promise of it.

The async iteration protocol is the same shape as before, with three swaps:

  1. Use Symbol.asyncIterator instead of Symbol.iterator.
  2. next() returns a promise that resolves to { done, value }. Marking it async is the easy way to get that, and it lets you await inside.
  3. Loop with for await (let item of iterable) — the await keyword is the whole difference.
sync
next() → { value, done }
async
next() → Promise → { value, done }
Sync vs async: the async iterator hands back a Promise that later resolves to the result object.

Take the same floors, but now deliver a value once per second:

let floors = {
  bottom: 2,
  top: 6,

  [Symbol.asyncIterator]() { // (1)
    return {
      current: this.bottom,
      last: this.top,

      async next() { // (2)

        // the await here is the point — next() can now pause
        await new Promise(resolve => setTimeout(resolve, 1000)); // (3)

        if (this.current <= this.last) {
          return { done: false, value: this.current++ };
        } else {
          return { done: true };
        }
      }
    };
  }
};

(async () => {

  for await (let value of floors) { // (4)
    alert(value); // 2, 3, 4, 5, 6 — one per second
  }

})()

Walking the labels:

  1. The object exposes Symbol.asyncIterator (1) instead of the plain one.
  2. Its iterator’s next() returns a promise (2). It doesn’t strictly have to be async — any method that returns a promise satisfies the protocol — but async gives you await for free, which is why it’s the natural choice. Here we just pause a second (3).
  3. You loop with for await (let value of floors) (4). That calls floors[Symbol.asyncIterator]() once, then awaits each next().

The differences fit in a small table:

Iterators Async iterators
Object method to provide iterator Symbol.iterator Symbol.asyncIterator
next() return value is any value a Promise
to loop, use for..of for await..of

Reading about for await is one thing; watching values trickle in on a timer makes the “iteration that waits” idea click. Press the button — each value appears only after the previous next() finishes its pause. Try changing the delay or the to value in the code:

interactiveAn async-iterable set of floors, one value at a time

A refresher on generators

Writing iterators by hand is verbose. Generators collapse most of that boilerplate, which is why in practice they’re how you usually make something iterable.

A generator function is written function* (the star matters), and each yield hands out one value. for..of pulls those values.

function* countRange(start, end) {
  for (let i = start; i <= end; i++) {
    yield i;
  }
}

for (let value of countRange(10, 14)) {
  alert(value); // 10, 11, 12, 13, 14
}

The engine does the next() / { done, value } bookkeeping for you. Every yield is a { done: false, value }; when the function returns, you get { done: true }.

Change the two numbers and re-run to see the same generator drive different sequences — and notice what happens when from is larger than to (the loop body never runs, so for..of collects nothing):

interactiveA generator function fed by for..of

You can plug a generator straight into Symbol.iterator. Compare the placeholder version:

let floors = {
  bottom: 2,
  top: 6,
  [Symbol.iterator]() {
    return <object with next to make floors iterable>
  }
}

…with the generator version, which drops all the manual state:

let floors = {
  bottom: 2,
  top: 6,

  *[Symbol.iterator]() { // shorthand for [Symbol.iterator]: function*()
    for (let value = this.bottom; value <= this.top; value++) {
      yield value;
    }
  }
};

for (let value of floors) {
  alert(value); // 2, 3, 4, 5, 6
}

No current, no last, no next — the loop variable is the state, and the engine remembers where it paused. The Generators chapter goes deeper.

There’s a hard limit, though: a regular generator can’t await. Its values must be ready synchronously because for..of won’t wait. So how do you generate values from, say, a network request? That’s the last piece.

Async generators

Prepend async to function* and the generator becomes asynchronous. Now yield can hand out values that took time to produce, and you drive it with for await.

async function* countRange(start, end) {

  for (let i = start; i <= end; i++) {

    // now await is allowed inside the generator
    await new Promise(resolve => setTimeout(resolve, 1000));

    yield i;
  }

}

(async () => {

  let generator = countRange(10, 14);
  for await (let value of generator) {
    alert(value); // 10, 11, 12, 13, 14 — with a pause between each
  }

})();

Because the generator is async, its body can await promises, hit the network, wait on timers — whatever. Each yield produces the next value only after that work finishes.

t=0sawait 1s→yield 10→await 1s→yield 11→ … →yield 14→done
An async generator yields on its own clock; for await..of waits for each value.

Making floors async-iterable with a generator

Just as a regular generator can back Symbol.iterator, an async generator can back Symbol.asyncIterator. Same floors, but each value now arrives a second apart:

let floors = {
  bottom: 2,
  top: 6,

  // same as [Symbol.asyncIterator]: async function*()
  async *[Symbol.asyncIterator]() {
    for (let value = this.bottom; value <= this.top; value++) {

      // pause between values — wait for something to be ready
      await new Promise(resolve => setTimeout(resolve, 1000));

      yield value;
    }
  }
};

(async () => {

  for await (let value of floors) {
    alert(value); // 2, 3, 4, 5, 6
  }

})();

Compare this against the hand-written async iterator earlier in the chapter — same behavior, a fraction of the code. The async *[Symbol.asyncIterator]() shorthand is the async twin of *[Symbol.iterator]().

Real-life example: paginated data

The examples so far were toys built to show the syntax. Here’s where async generators earn their keep.

Plenty of APIs hand back data in pages. Ask for a list of users and you get a fixed batch — say 100 — plus a pointer to the next page. Fetch that, get the next batch and the next pointer, and so on until there are no more pages. The shape is the same whether it’s users, products, or log entries.

GitHub’s commits API works this way:

  • You request https://api.github.com/repos/<repo>/commits.
  • It replies with a JSON array of commits and a Link response header pointing at the next page.
  • You follow that link for the next batch, and repeat.
page 1
30 commits
Link → page 2
→
page 2
30 commits
Link → page 3
→
…
→
last page
N commits
no Link → stop
Pagination as a chain: each response carries the URL of the next page until the Link header runs out.

The consumer of your code shouldn’t have to know any of that. The goal is for calling code to look like this — no headers, no page tracking, just commits:

for await (let commit of streamCommits("username/repository")) {
  // process commit
}

An async generator hides all the pagination behind that loop:

async function* streamCommits(repo) {
  let url = `https://api.github.com/repos/${repo}/commits`;

  while (url) {
    const response = await fetch(url, { // (1)
      headers: {'User-Agent': 'codex-commit-reader'}, // GitHub requires a User-Agent header
    });

    const body = await response.json(); // (2) parse JSON (an array of commits)

    // (3) the next page URL lives in the Link header — pull it out
    let nextPage = response.headers.get('Link').match(/<(.*?)>; rel="next"/);
    nextPage = nextPage?.[1];

    url = nextPage;

    for (let commit of body) { // (4) yield commits one at a time
      yield commit;
    }
  }
}

How the pieces fit:

  1. fetch downloads one page (1). The first URL is the commits endpoint; every later URL comes from the previous response’s Link header. fetch lets you attach headers — GitHub needs a User-Agent. See the Fetch chapter for the details.
  2. The body is JSON, so await response.json() gives you the array of commits (2).
  3. The next page URL sits inside the Link header in a specific format, so a regular expression digs it out (3). A typical result looks like https://api.github.com/repositories/41028493/commits?page=2, generated by GitHub. Regular expressions are covered in the Regular expressions chapter. The ?.[1] guards against there being no next page — when the match is null, nextPage becomes undefined, the while (url) test fails, and the loop ends.
  4. Each commit is yielded individually (4). When the current page’s commits run out, the inner for finishes and the outer while fires another request — but only if there’s still a url.

Using it, printing the first hundred authors:

(async () => {

  let count = 0;

  for await (const commit of streamCommits('maya-labs/aurora')) {

    console.log(commit.author.login);

    if (++count == 100) { // stop after 100 commits
      break;
    }
  }

})();

// If you run this in an external sandbox, paste the streamCommits function above first.

Breaking out of the loop at 100 stops the generator too — no wasted requests for pages you’ll never read. From the caller’s side there’s no pagination at all, just an async generator that yields commits on demand.

The real API needs the network, which this in-browser sandbox can’t reach — so the demo below stands in a fake paginated source where each “page” resolves after a short delay. The structure is identical to streamCommits: an async generator loops through pages, yielding one author at a time. Watch the page banners appear as each request resolves, and note that breaking at five authors means the third page is never fetched:

interactiveAn async generator hiding pagination behind one loop

Summary

Ordinary iterators and generators are for data that’s already here. When values arrive over time — with delays, from the network — reach for their async counterparts and swap for..of for for await..of.

The iterator-side differences:

Iterable Async Iterable
Method to provide iterator Symbol.iterator Symbol.asyncIterator
next() return value is {value:…, done:…} Promise that resolves to {value:…, done:…}

The generator-side differences:

Generators Async generators
Declaration function* async function*
next() return value is {value:…, done:…} Promise that resolves to {value:…, done:…}

Streaming data — a large file downloading or uploading chunk by chunk — is a natural fit for async generators. Browsers also ship a separate Streams API with its own interfaces for transforming a stream and piping one into another, for example downloading from one place and forwarding elsewhere as the bytes arrive.