Asyncio: Start a non-blocking listening server

Question

This is the basic tcp server from asyncio tutotial:

import asyncio class EchoServerClientProtocol(asyncio.Protocol): def connection_made(self, transport): peername = transport.get_extra_info('peername') print('Connection from {}'.format(peername)) self.transport = transport def data_received(self, data): message = data.decode() print('Data received: {!r}'.format(message)) print('Send: {!r}'.format(message)) self.transport.write(data) print('Close the client socket') self.transport.close() loop = asyncio.get_event_loop() # Each client connection will create a new protocol instance coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888) server = loop.run_until_complete(coro) # Serve requests until CTRL+c is pressed print('Serving on {}'.format(server.sockets[0].getsockname())) try: loop.run_forever() except KeyboardInterrupt: pass # Close the server server.close() loop.run_until_complete(server.wait_closed()) loop.close()

Like all (i found) other examples it uses blocking loop.run_forever().
How do i start listeting server and do something else in the time?
I have tried to outsource starting server in a function and start this function with asyncio.async(), but with no success. What i'm missing here?

What other thing do you want to do? Normally, everything your application does happens inside the event loop; so you'd use loop.call_soon to schedule a callback prior to the loop starting, in addition to starting the server, prior to making the call to loop.run_forever. Would that cover your use-case, or do you want to run some other code that isn't interacting with asyncio at all? — dano
– dano, Commented Jul 31, 2015 at 13:50
my use case is here: stackoverflow.com/questions/31727547/… — Stefan Weiss
– Stefan Weiss, Commented Aug 3, 2015 at 6:52
It's not really clear from that question what the "other thing" you want to do is. What is it you need added to the example code you included above? — dano
– dano, Commented Aug 3, 2015 at 14:21
my basic question is: how to start non-blocking listening server with python3 for client-server communication over network? my first approach was to start a second process, listen all the time and write incoming messages in multiprocessing.queue. BUT (i thought) maybe is a lib out there to do it easier and "nicier"? — Stefan Weiss
– Stefan Weiss, Commented Aug 4, 2015 at 10:38

Martin Richard · Accepted Answer · 2015-08-06 14:16:31Z

You can schedule several concurrent asyncio tasks before calling loop.run_forever().

@asyncio.coroutine def other_task_coroutine(): pass # do something start_tcp_server_task = loop.create_task(loop.create_server( EchoServerClientProtocol, '127.0.0.1', 8888)) other_task = loop.create_task(other_task_coroutine()) self.run_forever()

When you call loop.create_task(loop.create_server()) or loop.create_task(other_task_coroutine()), nothing is actually executed: a coroutine object is created and wrapped in a task (consider a task to be a shell and the coroutine an instance of the code that will be executed in the task). The tasks are scheduled on the loop when created.

The loop will execute start_tcp_server_task first (as it's scheduled first) until a blocking IO event is pending or the passive socket is ready to listen for incoming connections.

You can see asyncio as a non-preemptible scheduler running on one CPU: once the first task interrupts itself or is done, the second task will be executed. Hence, when one task is executed, the other one has to wait until the running task finishes or yields (or "awaits" with Python 3.5). "yielding" (yield from client.read()) or "awaiting" (await client.read()) means that the task gives back the hand to the loop's scheduler, until client.read() can be executed (data is available on the socket).

Once the task gave back the control to the loop, it can schedule the other pending tasks, process incoming events and schedule the tasks which were waiting for those events. Once there is nothing left to do, the loop will perform the only blocking call of the process: sleep until the kernel notifies it that events are ready to be processed.

In this context, you must understand that when using asyncio, everything running in the process must run asynchronously so the loop can do its work. You can not use multiprocessing objects in the loop.

Note that asyncio.async(coroutine(), loop=loop) is equivalent to loop.create_task(coroutine()).

This is not exactly that i'm hoped for, but thank you for clarifying. I think i will use rabbitmq/pika for my purpose

Henry · Accepted Answer · 2016-11-18 02:17:58Z

Additionally, you can consider running what you want in an executor. For example.

coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888) server = loop.run_until_complete(coro) async def execute(self, loop): await loop.run_in_executor(None, your_func_here, args: asyncio.async(execute(loop)) loop.run_forever()

An executor will run whatever function you want in an executor, which wont block your server.

Collectives™ on Stack Overflow

Asyncio: Start a non-blocking listening server

2 Answers 2

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Linked

Related