6

I have observed that the asyncio.run_coroutine_threadsafe function does not accept general awaitable objects, and I do not understand the reason for this restriction. Observe

import asyncio async def native_coro(): return @asyncio.coroutine def generator_based_coro(): return class Awaitable: def __await__(self): return asyncio.Future() loop = asyncio.get_event_loop() asyncio.run_coroutine_threadsafe(native_coro(), loop) asyncio.run_coroutine_threadsafe(generator_based_coro(), loop) asyncio.run_coroutine_threadsafe(Awaitable(), loop)

Running this with Python 3.6.6 yields

Traceback (most recent call last): File "awaitable.py", line 24, in <module> asyncio.run_coroutine_threadsafe(Awaitable(), loop) File "~/.local/python3.6/lib/python3.6/asyncio/tasks.py", line 714, in run_coroutine_threadsafe raise TypeError('A coroutine object is required') TypeError: A coroutine object is required

where line 24 is asyncio.run_coroutine_threadsafe(Awaitable(), loop).

I know I can wrap my awaitable object in a coroutine defined like

awaitable = Awaitable() async def wrapper(): return await awaitable asyncio.run_coroutine_threadsafe(wrapper(), loop)

however my expectation was that the awaitable would be a valid argument directly to run_coroutine_threadsafe.

My questions are:

  1. What is the reason for this restriction?
  2. Is the wrapper function defined above the most conventional way to pass an awaitable to run_coroutine_threadsafe and other APIs that demand an async def or generator-defined coroutine?
Improve this question

1 Answer 1

Reset to default
8

What is the reason for this restriction?

Looking at the implementation, the reason is certainly not technical. Since the code already invokes ensure_future (rather than, say, create_task), it would automatically work, and work correctly, on any awaitable object.

The reason for the restriction can be found on the tracker. The function was added in 2015 as a result of a pull request. In the discussion on the related bpo issue the submitter explicitly requests the function be renamed to ensure_future_threadsafe (in parallel to ensure_future) and accept any kind of awaitable, a position seconded by Yury Selivanov. However, Guido was against the idea:

I'm against that idea. I don't really see a great important future for this method either way: It's just a little bit of glue between the threaded and asyncio worlds, and people will learn how to use it by finding an example.

[...]

But honestly I don't want to encourage flipping back and forth between threads and event loops; I see it as a necessary evil. The name we currently have is fine from the POV of someone coding in the threaded world who wants to hand off something to the asyncio world.

Why would someone in the threaded world have an asyncio.future that they need to wait for? That sounds like they're mixing up the two worlds -- or they should be writing asyncio code instead of threaded code.

There are other comments in a similar vein, but the above pretty much sums up the argument.

Is the wrapper function defined above the most conventional way to pass an awaitable to run_coroutine_threadsafe and other APIs that demand an async def or generator-defined coroutine?

If you actually need a coroutine object, something like wrapper is certainly a straightforward and correct way to get one.

If the only reason you're creating the wrapper is to call run_coroutine_threadsafe, but you're not actually interested in the result or the concurrent.futures.Future returned by run_coroutine_threadsafe, you can avoid the wrapping by calling call_soon_threadsafe directly:

loop.call_soon_threadsafe(asyncio.ensure_future, awaitable)
Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

Nicely put. To give an example why "someone in the threaded world would have an [awaitable] that they need to wait for", I am testing a websocket server and I have multiple websockets clients executing concurrently with test steps executed using Robot Framework. Not having control over the execution context means I cannot make it all async. In my case I do care about the concurrent.futures.Future, but that use for asyncio.ensure_future will be handy elsewhere.
@ChrisHunt I hear you. IMO the functionality provided by run_coroutine_threadsafe (and run_in_executor in the other direction) is a fantastic glue between asyncio and the classic synchronous/threading code, very important for introducing asyncio to an existing code base. It's unfortunate that it's underrated, often being omitted from asyncio introductory materials. I understand that the intention was to encourage as much code as possible to be "native" coroutine-based code, but that cannot happen overnight, and until it does, there needs to be a convenient way for the two to interoperate.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.