I need to process several pages of data from server. I would like to make a generator for it like this. Unfortunately I get TypeError: 'async_generator' object is not iterable
async def get_data(): i = 0 while i < 3: i += 1 data = await http_call() # call to http-server here yield data data = [i for i in get_data()] # inside a loop Next variant raises TypeError: object async_generator can't be used in 'await' expression
data = [i for i in await get_data()] # inside a loop Use async for in your comprehension. See PEP 530 -- Asynchronous Comprehensions
data = [i async for i in get_data()] Depending on what version of Python you're using, this may only be available in async def functions.
async doesn't mean "concurrent" as in preemptive multithreading. There is no preemptive multithreading at the pure-Python level, thanks to the GIL. async means cooperative multitasking, meaning the developer explicitly relinquishes control to other currently paused coroutines and asynchronous generators at each await expression. In the absence of explicit multithreading or multiprocessing, asynchronous generators still yield their results sequentially (i.e., non-concurrently).asyncio.Queue class and asyncio.as_completed() function for I/O-bound operations or the async.run_in_executor() function for CPU-bound operations instead.