Timeline for How to deal with the zombie apocalypse caused by Python ExternalSessions?
Current License: CC BY-SA 4.0
27 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| S Aug 6 at 20:03 | history | bounty ended | Henrik Schumacher | ||
| S Aug 6 at 20:03 | history | notice removed | Henrik Schumacher | ||
| Aug 4 at 15:18 | comment | added | Daniel Lichtblau | “[N]ot crashing the kernel” can be an elusive goal even for the most experienced. Here the problem is compounded by having even less control over the behavior of external libraries or the wrath they might show to the kernel that invoked them. This is especially true when LibraryLink is thrown into the mix. | |
| S Aug 3 at 16:36 | history | bounty started | Henrik Schumacher | ||
| S Aug 3 at 16:36 | history | notice added | Henrik Schumacher | Reward existing answer | |
| Aug 3 at 16:29 | vote | accept | Henrik Schumacher | ||
| Aug 1 at 19:27 | comment | added | Henrik Schumacher | @JasonB Yes, I do that every minute every day. But when you develop code in C++, then some segfaults and other unpleasantries are unavoidable. | |
| Aug 1 at 18:47 | comment | added | Jason B. | Have you tried not crashing the kernel? | |
| Aug 1 at 16:16 | answer | added | b3m2a1 | timeline score: 7 | |
| Jul 30 at 22:27 | comment | added | b3m2a1 | @HenrikSchumacher you can create a python Thread in the script you run that gets initialized with the WolframKernel PID when it's launched and if that PID is killed it terminates, this wouldn't be too much work depending on how you're calling your external libraries | |
| Jul 30 at 19:35 | comment | added | Michael E2 | @b3m2a1 Right, in Unix, a parent doesn't kill its children when it dies. It can trap some signals and kill its children before exiting, but it cannot trap a SIGKILL signal. It can trap SIGSEGV (segmentation fault), though. Proceeding after SIGSEGV seems risky. SIGKILL normally isn't sent; it should be SIGTERM, unless the process ignores SIGTERM. It's possible that the Wolfram Kernel traps these signals (except SIGKILL) and does its own cleanup. Then maybe $Epilog could work. -- Nope, it seems not, using kill.... | |
| Jul 30 at 17:56 | comment | added | Henrik Schumacher | Maybe one can launch a separate background process that listens for the WolframKernel and then kills the Pythons processes... But I had hope that there might be an easier solution. | |
| Jul 30 at 17:53 | history | edited | Henrik Schumacher | CC BY-SA 4.0 | added 56 characters in body |
| Jul 30 at 17:49 | comment | added | Henrik Schumacher | Yeah, I learnt that, too, from a quick research. People suggest to let the child process ping the parent continuously and to commit suicide when the ping is not answered. But I don't have an idea how to really facilitate this. | |
| Jul 30 at 17:31 | comment | added | b3m2a1 | Just coming back in to say that this appears to be surprisingly tricky in general on Unix systems...I can't find an easy way to force a child process to die if its parent gets killed without cleanup, outside of throwing in something on the python side that takes the kernel PID and exits if that job has died | |
| Jul 30 at 16:53 | comment | added | Henrik Schumacher | @MichaelE2 Yes exactly. The problem is that the Mma kernel cannot run cleanup code because it is shut down from the outside (by the operating system). So $Epilog won't help I am afraid. | |
| Jul 30 at 16:52 | comment | added | b3m2a1 | On my version of Mathematica, you can potentially temporarily Block to modify ExternalEvaluate`Private`$SessionInitilizationRules to make StartProcess start a daemon process appropriately by tweaking the "Process" key | |
| Jul 30 at 16:51 | comment | added | Henrik Schumacher | @ b3m2a1 Well, I typically try my best to work around 3 (it only happens when I make a mistake). But I can do little about 1. and 2., and I have little control over what happens on the user's system (although I should have ruled out 1. and 2. by then, of course). | |
| Jul 30 at 16:26 | comment | added | b3m2a1 | Is this partially a front-end issue? You could partially work around (3) by controlling the logging behavior as the FE can be fragile | |
| Jul 30 at 15:44 | comment | added | David G. Stork | This site is read by students and scholars from innumerable countries and languages, so while I as an English speaker am amused by the title, I'm certain that there are many whose native tonge is Tagalog, or Urdu, or Mandarin, or ... are confused by the title. I would change it. | |
| Jul 30 at 14:58 | comment | added | Michael E2 | There's $Epilog. See also the docs for Quit[]. Also DefineInputStreamMethod[]. I don't expect them to handle a KILL signal (on Unix-type systems). I don't know if the other reasons for dying follow the regular cleanup protocol for quitting the kernel. You might have to assume that the kernel can die unable to do any cleanup. It is often recommended that child process periodically check on their parent. In Unix, streams are closed when a process dies, for instance. | |
| Jul 30 at 14:46 | history | edited | Henrik Schumacher | CC BY-SA 4.0 | deleted 1457 characters in body |
| Jul 30 at 13:43 | comment | added | Henrik Schumacher | Come one, zombie process is a well-defined technical term. | |
| Jul 30 at 13:40 | comment | added | Jason B. | The title is great for a creative writing exercise I guess, not as clear and descriptive as a more straightforward title would be though. | |
| Jul 30 at 13:09 | comment | added | Henrik Schumacher | +1 for your user name! XD | |
| Jul 30 at 13:08 | comment | added | Hans Olo | +1 just for the title! | |
| Jul 30 at 13:02 | history | asked | Henrik Schumacher | CC BY-SA 4.0 |