I'm using Django and Celery for one async task. I need to know how to kill a Celery task correctly.
For example, I have this Celery task in a Django view:
... sometask.delay(some1, some2) ... To kill this "sometask" I can use a sys.exit() inside the task? Example:
@shared_task def sometask(some1, some2): ... ... if some1 == False: sys.exit('Abort!') This sys.exit() will work inside a Celery task?
Best Regards,
You mention 'kill a celery task' and you also mention sys.exit(). Generally killing a process involves some external interference to terminate the process. Using sys.exit() would mean the process is terminated from within.
Considering this, here are few options:
maxtasksperchild parameter of the worker node.If you want to end the celery task from outside the process:
a) If you know the taskid you can try to revoke the task.
b) If you want to 'kill' (TERM) the worker process that is executing a task, then it's a different thing. You will need to handle relevant signals. I think a TERM signal is sent only to the main worker process and not the pool worker processes. That's what I remember because I was trying to solve a somewhat similar problem.
From the way you have tried to use sys.exit() in your code, I think #1 is all you need.
from worker's guide :
When a worker receives a revoke request it will skip executing the task, but it won’t terminate an already executing task unless the terminate option is set.
If terminate is set the worker child process processing the task will be terminated
using app.control.revoke(task_id,terninate=True),this will kill the worker's subprocess and all the other tasks will be fine.
here i am starting two tasks and terminate one of them:
app.control.revoke("c028bd99-8394-4e2c-86ab-440ca2290d67",terninate=True) [2015-11-19 04:55:09,458: INFO/MainProcess] Received task: celery.starmap[73c53773-2dd5-40d0-b9a0-83b2f12eba9b] [2015-11-19 04:55:09,465: INFO/MainProcess] Received task: celery.starmap[c028bd99-8394-4e2c-86ab-440ca2290d67] [2015-11-19 04:57:13,993: INFO/MainProcess] Tasks flagged as revoked: c028bd99-8394-4e2c-86ab-440ca2290d67 [2015-11-19 04:59:32,684: INFO/MainProcess] Terminating c028bd99-8394-4e2c-86ab-440ca2290d67 (15) [2015-11-19 04:59:32,781: ERROR/MainProcess] Task celery.starmap[c028bd99-8394-4e2c-86ab-440ca2290d67] raised unexpected: Terminated(15,) Traceback (most recent call last): File "....../lib/python2.7/site-packages/billiard/pool.py", line 1678, in _set_terminated raise Terminated(-(signum or 0)) Terminated: 15 
