Python 3.5 Gets async and await: How Native Coroutines Work

Python 3.5.0 came out last Sunday, September 13, right on the date the PEP 478 release schedule had planned for it. There’s plenty in the release (the @ operator for matrix multiplication, os.scandir(), the new typing module), but the change I expect to show up in the most new code is PEP 492, which adds async def and await to the language. If you’ve written asyncio code on 3.4, the ideas will be familiar. What’s new is that coroutines are now a real part of Python instead of a clever use of generators.

Coroutines before 3.5

Python has done coroutines for years by bending generators. You could send values into a generator, yield from let one generator delegate to another, and asyncio, added in 3.4, built its whole model on top of that. A coroutine in 3.4 style looks like this:

import asyncio

@asyncio.coroutine
def slow_double(n):
    yield from asyncio.sleep(1)
    return n * 2

It works, but PEP 492 lays out why it’s awkward. Coroutines and ordinary generators share the same syntax, so they’re easy to confuse. Whether a function is a coroutine at all depends on whether a yield or yield from happens to appear somewhere in its body, so adding or removing one during a refactor can change what the function is and cause confusing bugs. And because you can only suspend where yield is allowed, there was no clean way to do asynchronous work while entering or leaving a with block or while fetching the next item in a for loop.

async def and await

Here’s the same coroutine in 3.5:

async def slow_double(n):
    await asyncio.sleep(1)
    return n * 2

A function defined with async def is always a coroutine function, even if it never awaits anything. Calling it doesn’t run the body. You get a coroutine object back, and nothing happens until something awaits it or schedules it on an event loop. Putting yield inside an async def is a SyntaxError in 3.5, and so is using await outside one. Native coroutine objects also aren’t iterators, so you can’t accidentally loop over one.

await works like yield from with an extra check: it only accepts an awaitable. That’s a native coroutine, a generator based coroutine marked with the new types.coroutine() function (which @asyncio.coroutine now uses under the hood), or an object with an __await__ method that returns an iterator. asyncio’s Future got an __await__ method, so futures and tasks can be awaited directly. The PEP also promises full backwards compatibility with existing asyncio code, and the two styles work together in both directions. That matters, because a lot of asyncio itself, asyncio.sleep() included, is still written as decorated generators in the 3.5.0 source.

The grammar is a little friendlier too. await binds tighter than the arithmetic operators, so await fut + 1 awaits first and then adds, while yield from a() + b() tries to yield from the sum, which is almost never what you meant.

One thing to know from What’s New in Python 3.5: async and await aren’t full keywords yet. Under PEP 492 the tokenizer only treats them specially inside an async def, so older code that uses them as names still runs. They’re scheduled to become proper keywords in 3.7, though, so if you have a variable called async somewhere, now is a good time to rename it.

async with and async for

These two statements exist because some setup, teardown and iteration needs to wait on I/O. An asynchronous context manager defines __aenter__ and __aexit__, both returning awaitables, and async with awaits them on the way in and on the way out. The PEP’s examples are database transactions and locks, where the old with (yield from lock): becomes async with lock:.

async for does the same for loops. The iterator’s __anext__ method returns an awaitable, and the loop ends when it raises the new StopAsyncIteration exception instead of StopIteration. That lets something like a database cursor fetch the next batch of rows over the network halfway through a loop. asyncio’s StreamReader supports it in 3.5, which is how the What’s New example reads lines from a socket with async for line in reader. Like await, both statements only work inside an async def, and the coroutines section of the language reference has the exact code each one expands to.

How the event loop runs them

A coroutine object does nothing on its own. asyncio’s event loop drives it by wrapping it in a Task. loop.run_until_complete() does that for you when you give it a coroutine, and asyncio.ensure_future() or loop.create_task() do it explicitly. (ensure_future is the new name for asyncio.async(), which is deprecated in 3.5.) The 3.5 tasks and coroutines docs are clear that whatever you schedule only runs while the loop is running.

Underneath, it’s still generator machinery. The Task calls send() on the coroutine, and the coroutine runs ordinary Python until it awaits something that isn’t ready. At the bottom of that chain of awaits sits a Future, which yields itself up to the Task. The Task adds a callback to the Future and returns, so the loop can move on to anything else that’s ready. When the Future completes, because a timer fired or a socket became readable, the callback schedules the Task again, and the Task sends the result back into the coroutine right where it paused. asyncio.sleep() is a nice small example: in the 3.5.0 source it creates a Future, asks the loop to set its result after the delay with call_later(), and waits on it.

All of this happens on one thread. The docs spell it out: an event loop runs one task at a time, and a task only gives up control when it waits. The concurrency comes from lots of tasks spending most of their lives suspended, not from anything running in parallel.

Lots of waits at once

Here’s a small program that runs 100 fake network calls, each taking one second, using only what’s in 3.5:

import asyncio
import time

async def fake_request(n):
    # Stand-in for a network call that takes one second.
    await asyncio.sleep(1)
    return n * 2

async def main():
    tasks = [asyncio.ensure_future(fake_request(n)) for n in range(100)]
    return await asyncio.gather(*tasks)

loop = asyncio.get_event_loop()
start = time.monotonic()
results = loop.run_until_complete(main())
print(len(results), "results in", round(time.monotonic() - start, 2), "seconds")
loop.close()

ensure_future schedules all 100 tasks up front, and gather gives back a future whose result is the list of return values in the order you passed the tasks in, not the order they finished. Each task reaches its await, parks, and lets the next one start, so all 100 timers run at the same time and the program finishes in about one second instead of 100. gather would wrap bare coroutines for you, but the explicit ensure_future makes the scheduling visible.

When async doesn’t help

That speedup comes from the waiting, not the work. If a coroutine spends its time computing, say resizing images or hashing passwords, there’s nothing to wait on. It never reaches an await that suspends, so the loop can’t run anything else until it’s done, and every other task just sits there. For CPU bound work, async gives you nothing over plain sequential code. The asyncio docs point to loop.run_in_executor() for running a function in a pool of threads or processes instead. For heavy pure Python computation, a process pool is the one that helps, since CPython’s global interpreter lock lets only one thread execute Python bytecode at a time.

The classic mistake: blocking inside a coroutine

The mirror image of that problem bites people who really do have I/O. Change one line in the example:

async def fake_request(n):
    time.sleep(1)  # blocks the whole event loop
    return n * 2

It still runs and still returns the right answers, but now it takes about 100 seconds. time.sleep() knows nothing about the event loop. It blocks the one thread that the loop and every task share, so the tasks end up running one after another. The same goes for a synchronous HTTP client or a database driver that wasn’t written for asyncio. The Develop with asyncio page says it directly: don’t call blocking functions, because a call that blocks for a second delays every other task by a second.

The fix is to use something that cooperates with the loop, like asyncio’s own streams for sockets or asyncio.sleep() for delays, or to hand the blocking call off with await loop.run_in_executor(None, blocking_function, arg). Passing None uses the loop’s default thread pool, which the 3.5.0 source creates with five worker threads, so treat it as a patch for the occasional legacy call rather than a way to scale.

Two debugging aids make these mistakes easier to catch. Setting the PYTHONASYNCIODEBUG environment variable to 1 turns on asyncio’s debug mode, which among other checks logs any callback that runs for longer than 100 milliseconds, and a task stuck in time.sleep() shows up there. And if you forget the await entirely and just call fake_request(n), 3.5 emits a RuntimeWarning saying the coroutine was never awaited once the object is garbage collected, which beats silently doing nothing.

Where this leaves you

Nothing forces a rewrite. Code using @asyncio.coroutine and yield from keeps working in 3.5, and the asyncio docs recommend async def for new code that doesn’t need to run on older Pythons. My take is that the syntax is the smaller win. The bigger one is that a coroutine is now something you declare, the interpreter can tell apart from a generator, and tools can reason about. Reach for it when your program spends its time waiting on lots of sockets, and keep blocking calls and heavy computation off the loop’s thread.