Launch a totally independent process from Python

Question

I'm trying to launch a completely independent process from python. I can't use something simple like os.startfile since I need to pass arguments. Currently I'm using subprocess.popen which gets me 90% of the way there.

args = ["some_exe.exe", "some_arg", "another_arg"] subprocess.Popen(args, creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)

Using popen with detached creation flags & pipes for std* does start a new process that lives after the parent process dies. So thats all good. The problem is that the new 'child' process still holds a phantom handle to the parent. So if I try to uninstall the parent exe (my python script is bundled into an exe via pyinstaller) msiexec complains that the parent exe is still in use.

So the goal is to spawn a totally independent process to run "some_exe.exe" that doesn't have any handle back to the original process.

Note: This is for Windows XP and above. I'm developing on Win7.

Have you looked at stackoverflow.com/questions/204017/… and stackoverflow.com/questions/1196074/… ? I have no time to try it out for myself... — MartinStettner
– MartinStettner, Commented Nov 27, 2012 at 20:16
I looked at that post but I ended up using popen because, as one of the comments says, it is the new preferred way (according to docs. I have no idea why). However you're right I should (and am about to) try spawnl too. — greenhat
– greenhat, Commented Nov 27, 2012 at 20:38
Please report on the results of your experiments :) This might well be of interest to me in some future projects ... — MartinStettner
– MartinStettner, Commented Nov 27, 2012 at 20:41
huh... i'm running into crashes using spawnl (maybe why it is recommended not to use?). Some ppl on the earlier thread you linked were getting crashes too. Just running "os.spawnl(os.P_NOWAIT, r"c:\windows\notepad.exe")" from the python interpreter killed python for me. P_DETACH also gets a crash as does P_WAIT. — greenhat
– greenhat, Commented Nov 27, 2012 at 21:00
The issue may be with how your Python exe is packaged, e.g. it's maintaining a handle to one of the parent's DLLs -- something along those lines. You can examine the open handles with ProcessExplorer. Just speculation, but it's possible that pyinstaller is bundling some DLL that gets included on the PATH of the child process and therefore gets loaded. — ig0774
– ig0774, Commented Nov 27, 2012 at 21:35

Felipe · Accepted Answer · 2015-05-01 20:38:12Z

28

I think I found the answer. By using Popen with close_fds = True I was able to start up a process that was independent and without handles to the parent.

For docs look here and search for close_fds.

Or, on Windows, if close_fds is true then no handles will be inherited by the child process. Note that on Windows, you cannot set close_fds to true and also redirect the standard handles by setting stdin, stdout or stderr.

Note this solution only works on Windows. I have no idea about any *nix system.

edited May 1, 2015 at 20:38

Felipe

3,1592 gold badges30 silver badges46 bronze badges

answered Nov 27, 2012 at 21:46

greenhat

1,0811 gold badge12 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Zuzu Corneliu Over a year ago

I've been trying to start a daemon process using a python script called from within a makefile, but make.exe wouldn't exit until this daemon exited itself, which was not what I wanted. What worked eventually was a combination of close_fds=True and creationflags=DETACHED_PROCESS. Conclusion: use them both, e.g.: subprocess.Popen(<whatever>, close_fds=True, creationflags=DETACHED_PROCESS)

jtpereyda Over a year ago

In addition to @CorneliuZuzu's suggestion, I used CREATE_NEW_PROCESS_GROUP = 0x00000200 with DETACHED_PROCESS = 0x00000008: creationflags=DETACHED_PROCESS | CREATE_NEW_PROCESS_GROUP, close_fds=True

nirvana-msu Over a year ago

Note that starting from Python 3.7, on Windows it's no longer necessary to set close_fds=True - that is now the default and works together with redirecting stdin/stdout/stderr. Details in bug tracker here: bugs.python.org/issue19764

rstackhouse · Accepted Answer · 2017-07-27 19:49:38Z

I found this here:

On windows (win xp), the parent process will not finish until the longtask.py has finished its work. It is not what you want in CGI-script. The problem is not specific to Python, in PHP community the problems are the same.

The solution is to pass DETACHED_PROCESS Process Creation Flag to the underlying CreateProcess function in win API. If you happen to have installed pywin32 you can import the flag from the win32process module, otherwise you should define it yourself:

DETACHED_PROCESS = 0x00000008

pid = subprocess.Popen([sys.executable, "longtask.py"], creationflags=DETACHED_PROCESS).pid

I actually currently am using the detached flag (check out the code example). That flag does work to create a detached process but it still give the child a handle to the parent.

Collectives™ on Stack Overflow

Launch a totally independent process from Python

2 Answers 2

3 Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

2 Comments

Linked

Related