14

im spawning a script that runs for a long time from a web app like this:

os.spawnle(os.P_NOWAIT, "../bin/producenotify.py", "producenotify.py", "xx",os.environ) 

the script is spawned successfully and it runs, but till it gets over i am not able to free the port that is used by the web app, or in other words i am not able to restart the web app. how do i spawn off a process and make it completely independent of the web app?

this is on linux os.

2
  • 2
    Why aren't you using subprocess.Popen to do this? Commented Jun 9, 2009 at 21:21
  • 2
    S. Lott: will i be able to kill / restart the web app/ parent process after doing subprocess.Popen? thanks Commented Jun 10, 2009 at 2:41

2 Answers 2

26

As @mark clarified it's a Linux system, the script could easily make itself fully independent, i.e., a daemon, by following this recipe. (You could also do it in the parent after an os.fork and only then os.exec... the child process).

Edit: to clarify some details wrt @mark's comment on my answer: super-user privileges are not needed to "daemonize" a process as per the cookbook recipes, nor is there any need to change the current working directory (though the code in the recipe does do that and more, that's not the crucial part -- rather it's the proper logic sequence of fork, _exit and setsid calls). The various os.exec... variants that do not end in e use the parent process's environment, so that part is easy too -- see Python online docs.

To address suggestions made in others' comments and answers: I believe subprocess and multiprocessing per se don't daemonize the child process, which seems to be what @mark needs; the script could do it for itself, but since some code has to be doing forks and setsid, it seems neater to me to keep all of the spawning on that low-level plane rather than mix some high-level and some low-level code in the course of the operation.

Here's a vastly reduced and simplified version of the recipe at the above URL, tailored to be called in the parent to spawn a daemon child -- this way, the code can be used to execute non-Python executables just as well. As given, the code should meet the needs @mark explained, of course it can be tailored in many ways -- I strongly recommend reading the original recipe and its comments and discussions, as well as the books it recommends, for more information.

import os import sys def spawnDaemon(path_to_executable, *args) """Spawn a completely detached subprocess (i.e., a daemon). E.g. for mark: spawnDaemon("../bin/producenotify.py", "producenotify.py", "xx") """ # fork the first time (to make a non-session-leader child process) try: pid = os.fork() except OSError, e: raise RuntimeError("1st fork failed: %s [%d]" % (e.strerror, e.errno)) if pid != 0: # parent (calling) process is all done return # detach from controlling terminal (to make child a session-leader) os.setsid() try: pid = os.fork() except OSError, e: raise RuntimeError("2nd fork failed: %s [%d]" % (e.strerror, e.errno)) raise Exception, "%s [%d]" % (e.strerror, e.errno) if pid != 0: # child process is all done os._exit(0) # grandchild process now non-session-leader, detached from parent # grandchild process must now close all open files try: maxfd = os.sysconf("SC_OPEN_MAX") except (AttributeError, ValueError): maxfd = 1024 for fd in range(maxfd): try: os.close(fd) except OSError: # ERROR, fd wasn't open to begin with (ignored) pass # redirect stdin, stdout and stderr to /dev/null os.open(os.devnull, os.O_RDWR) # standard input (0) os.dup2(0, 1) os.dup2(0, 2) # and finally let's execute the executable for the daemon! try: os.execv(path_to_executable, args) except Exception, e: # oops, we're cut off from the world, let's just give up os._exit(255) 
Sign up to request clarification or add additional context in comments.

5 Comments

alex, it is a unix-like system. fedora linux. the web app does not have super user privileges. is that a problem? also the daemon should have the same working directory and environment variables as the webapp. how do i achieve these? thanks
alex, tx for clarification. i checked the recipe and it is little confusing. is that code in the parent web app? or does it go in the new script that is to be spawned off? thanks a lot again!
can I return the daemon pid to the function caller at the end of the function?
Why do you have two consecutive raise statements? I have always thought the second would never be executed.
In my application, the first child turns into a zombie with Alex' code. Before the "return", I inserted os.waitpid(pid, 0), which worked for me. But I'd appreciate if someone could confirm this.
12

You can use the multiprocessing library to spawn processes. A basic example is shown here:

from multiprocessing import Process def f(name): print 'hello', name if __name__ == '__main__': p = Process(target=f, args=('bob',)) p.start() p.join() 

7 Comments

this has the problem that the main process can not be killed till the subprocess is done. thanks!
Note multiprocessing library was added to Python 2.6. Here is documentation docs.python.org/library/multiprocessing.html
@mark If you want the main process to terminate immediately and not wait for the subprocesses, you just need to mark the subprocesses as daemons.
@CadentOrange "Additionally, these are not Unix daemons or services, they are normal processes that will be terminated (and not joined) if non-daemonic processes have exited." (documentation) doesn't that say that daemonic process will be killed in this scenario?
Isn't that what I said? If you mark the subprocesses as daemons, the subprocesses will be terminated immediately if the parent process (i.e. non-daemonic) is terminated.
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.