2

I'm encountering a subtle memory leak in a Python 3.12 application using asyncio, specifically when combining contextvars with deeply nested TaskGroup structures. The issue only appears under high concurrency and long runtime conditions.

Here’s a simplified version of the pattern:

import asyncio import contextvars user_context = contextvars.ContextVar("user_context") async def worker(): user_context.set("user123") await asyncio.sleep(0.1) async def nested_group(): async with asyncio.TaskGroup() as tg: for _ in range(100): tg.create_task(worker()) async def main(): for _ in range(1000): await nested_group() asyncio.run(main()) 

After running this for a while, memory usage steadily increases and never drops, even after all tasks complete. Profiling shows that contextvars are not being garbage collected as expected. I’ve ruled out circular references and confirmed that user_context is not being accessed outside the scope of each task.

What I’ve tried:

  1. Explicitly resetting the context variable after use.
  2. Using gc.collect() to force garbage collection.
  3. Replacing TaskGroup with manual task management — the leak disappears.

Questions:

  1. Is there a known issue with contextvars and TaskGroup interaction in Python 3.12?
  2. Could TaskGroup be retaining references to context states even after completion?
  3. Are there best practices for isolating context in high-concurrency async environments?

Any insights or suggestions would be greatly appreciated. This seems to be a rare edge case, and I haven’t found any related discussions or bug reports.

2
  • I will play along some experiments there, but at the first sight we have a "Could TaskGroup be retaining references to context states even after completion?" issue. Thank you for the minimal reproducible example: without that there would be no way this question could be approachable at all. Commented Jul 30 at 14:51
  • There were several fixes to the way asyncio handles internal data structures for Python 3.14- can you confirm the problem still happens in that version? The changes are described here: labs.quansight.org/blog/scaling-asyncio-on-free-threaded-python and one of the old structures could be messing up references to contextvars content. Commented Nov 19 at 17:11

0