10
$\begingroup$

I am working on a Mathematica package for knot theory that combines functionality of several other packages. Two notable examples are SnapPy and Regina. Fortunately, both have a rich Python interface and I can call them from Mathematica by using an ExternalSession.

Here is a minimal example that creates a Python environment that loads the packages snappy and numpy.

$snappy = StartExternalSession[Association[ "System" -> "Python", "Evaluator" -> <|"Dependencies" -> {"snappy"}, "EnvironmentName" -> "snappyenv"|> ]]; ExternalEvaluate[$snappy, " import numpy as np; import snappy; def snappy_setLink(pdcode) : global snappy_L a = np.array(pdcode); if a.size > 0 : snappy_L = snappy.Link(np.array(pdcode).tolist()) else: snappy_L = snappy.Link('0_1') "];

I could move on now and can define functions that send or retrieve link diagrams or compute some knot invariants. But these functions are not really important here. The point is that the Python session is alive now. On macOS the Activity Monitor shows me two(!) processes called python3.11, which together consume about 100 MB of RAM.

Now to my problem: My package creates such an ExternalSession whenever it is loaded. When I regularly Exit[] the kernel, they are deleted by the destructor of the ExternalSessionObject. But as I am still in the (never-ending) developing phase for this package and since I link a lot of experimental C++ code via LibraryLink, the following events are quite common in my workflow:

  1. the operating system shuts down the Mathematica kernel as mother process of the compiled library due to some segfault; or
  2. I have to shut down the Mathematica kernel manually because the library is trapped in an infinite loop; or
  3. --which is far more common--I have to shut down Mathematica altogether because the Mathematica frontend cannot swallow bazillions of log messages that are accidentally created; or
  4. something like that.

The problem with this not-so-uncommon kernel kills is that the Python processes are not killed this way. So after a couple of hours of work I have accumulated several dozens of Python zombie processes that happily gnaw away my brain RAM. Well I have a lot of it (of both, haha), but it is very annoying, nonetheless. Also, at some point I want to ship this package to customers and they might complain about the undead feasting on their hardware.

So, is there any canonical solution to this? The Python session is run as a child process of the Mathematica kernel WolframKernel. Is it possible to make the operating system automatically kill the child process if the parent is killed? Is there some magical option for StartExternalSession maybe? Also, are there cross-platform solutions to this?

Improve this question
$\endgroup$
18
  • 4
    $\begingroup$ +1 just for the title! $\endgroup$ Commented Jul 30 at 13:08
  • 2
    $\begingroup$ +1 for your user name! XD $\endgroup$ Commented Jul 30 at 13:09
  • 6
    $\begingroup$ Come one, zombie process is a well-defined technical term. $\endgroup$ Commented Jul 30 at 13:43
  • 3
    $\begingroup$ This site is read by students and scholars from innumerable countries and languages, so while I as an English speaker am amused by the title, I'm certain that there are many whose native tonge is Tagalog, or Urdu, or Mandarin, or ... are confused by the title. I would change it. $\endgroup$ Commented Jul 30 at 15:44
  • 2
    $\begingroup$ @b3m2a1 Right, in Unix, a parent doesn't kill its children when it dies. It can trap some signals and kill its children before exiting, but it cannot trap a SIGKILL signal. It can trap SIGSEGV (segmentation fault), though. Proceeding after SIGSEGV seems risky. SIGKILL normally isn't sent; it should be SIGTERM, unless the process ignores SIGTERM. It's possible that the Wolfram Kernel traps these signals (except SIGKILL) and does its own cleanup. Then maybe $Epilog could work. -- Nope, it seems not, using kill.... $\endgroup$ Commented Jul 30 at 19:35

1 Answer 1

Reset to default
7
+100
$\begingroup$

As Unix systems don't seem to have an easy mechanism for this, here's a targeted solution that requires minimal effort but does it on the python side. Basically we spin up a python Thread (not a real thread, more like a coroutine but also not a coroutine) that just checks if the Wolfram Kernel process that spawned it is dead and if so terminates itself. You can make this more graceful but the easy example was using SIGKILL

Here is the script I have

ExternalEvaluate[ $snappy, StringTemplate[ " import threading import psutil import time import signal import os WOLFRAM_PID = `` PYTHON_PID = `` def check_kill_process(w_pid, cur_pid): if not psutil.pid_exists(w_pid): os.kill(cur_pid, signal.SIGKILL) # maybe make this less dramatic exit(1) return True def listen_for_proc(w_pid, cur_pid, polling_time=5): while check_kill_process(w_pid, cur_pid): time.sleep(polling_time) def setup_parent_listener(): thread = threading.Thread( target=listen_for_proc, args=(WOLFRAM_PID, PYTHON_PID) ) thread.start() return thread # PUT YOUR SCRIPT HERE # put this in if __name __ == '__main __' setup_parent_listener() " ][$snappy["Process"]["PPID"], $snappy["Process"]["PID"]] ]

where we're directly injecting the relevant process IDs into the script for safety

The relevant chunk of python code is here

def check_kill_process(w_pid, cur_pid): if not psutil.pid_exists(w_pid): os.kill(cur_pid, signal.SIGKILL) # maybe make this less dramatic exit(1) return True

where w_pid and cur_pid are WOLFRAM_PID and PYTHON_PID. This works on my system if I kill -9 the WOLFRAM_PID. You could wrap this in a little module to make it a little bit less ugly too

Improve this answer
$\endgroup$
3
  • $\begingroup$ Cool, I have to look into this tomorrow. Just a question as the comment # PUT YOUR SCRIPT HERE" confuses me: Will this threadthat runs listen_for_proc wait in the background? My usecase is that I call into this Python session every now and then; I keep the session alive to save the startup costs. So the code I want to execute will land after the line setup_parent_listener() in the Python session. Will that work? $\endgroup$ Commented Aug 1 at 19:33
  • 1
    $\begingroup$ @HenrikSchumacher you can set the listener up beforehand. The thread will run in the background on the same process only when your main process isn't doing anything. I only put it at the end because your initial script was a bunch of function definitions so I assumed this was your startup script and then you'd call into those functions down the line $\endgroup$ Commented Aug 3 at 14:56
  • 2
    $\begingroup$ Awesome! The only thing I had to do to make it work was to add "psutil" to the "Dependencies" when I create the session with StartExternalSession. Works like a charm! Thank you, you helped me a lot! $\endgroup$ Commented Aug 3 at 16:30

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.