0

I built an infinite drawing app (similar to webcanvas) with django.

I made another small python app able to render areas of the canvas with cefpython.

My goal is to have background tasks on my server to render areas on the canvas which need to be updated (I would like to avoid slowing down my django server):

  • the main django app would add a render task to a queue whenever users are modifying the canvas,
  • the rendering app would use this queue and render the areas in a background thread.

What is the simplest way to do this?

Improve this question
3
  • Have you looked at Celery for this? celery.readthedocs.org/en/latest/django/… Commented Mar 23, 2015 at 16:26
  • An can also take a look at DjangoRQ wich is way simpler and faster to deploy than Celery Commented Mar 23, 2015 at 16:37
  • @dylrei I am looking at it, but I don't know if it is appropriate for my problem (Overkill? How will it be used?) I will detail my question once I read more. Commented Mar 23, 2015 at 16:53

1 Answer 1

Reset to default
1

I resolve same problem in such way:

  • Queue stored in PostgreSQL (I used it because it already was, PostgreSQL has enough performance for my purpose and I didn't want overcomplicate infrastructure).
  • Render are using multiprocessing module. It's running with upstart. I'm using upstart because it's easiest way to run separate app.

If you want to limit available resource for your application, you can setup limit in upstart: limit rss 231236400 231236400 (size in bytes), or run application using cpulimit.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

Thanks, upstart seems to be a good tool, but how can I set a low priority to my process (run in background mostly when my server is not busy)?
@arthur.sw, I recommend you to use nice. So process will be run with smallest priority.
nice enables to change the scheduling priority. I would prefer something like a CPU and/or memory usage limit.
@arthur.sw, so, you can run your app with cpulimit and set upstart limit: limit rss 231236400 231236400 (size in bytes).
After investigating the different possibilities, I decided to go with this method. You can make your answer more general though: I made a small script which 1. regularly read data from database (in my case it's mongoDB) 2. render images based on the data (multiprocessing and upstart is not necessary) It is possible to limit cpu usage with cpulimit.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.