3

I've got some Python scripts that are now being run by a wide variety of folks across multiple test platforms. This creates a problem for 2 reasons:

  1. The python3 path is not the same across all the testers, and
  2. Not all of the people running the scripts have a path to Python set in $PATH.

Because of (1) I can't hard-code the shebang since its tester dependent and because of (2) #!/usr/bin/env python3 isn't guaranteed to work if placed at the top of Python file.

I know that the python3 interpreter is going to be in one of a few locations across the testers. So what I'm wondering, is it possible to replace the #!/usr/bin/env python3 at the top of the Python file with a call to a bash shell script that looks for the python location and then "sets" it for the script? If that's not possible, then the rest of this is moot.

I created a bash script that does look through the possible locations until it finds the interpreter, but what I don't know how to do is return it in the top of a Python file.

For example, I created a basic python file (shebang.py)

#!./pyshebang.sh print("Hello World")

pyshebang.sh does 2 things, it appends the found python path to PATH, and echo's back that path to the interpreter. If I run the python script above, stdout gets the echo from the bash script, but not the print from the python script.

Improve this question
5
  • 1
    Not really; but it is not your script's job (or at least, not your job, as the author of the script) to know where the correct interpreter is located. That's the job of the installer, which is python setup.py ... replaces any shebang containing the word python (with #!python being the minimal example) with the path specified by the installer. Commented Feb 27, 2019 at 21:56
  • It's worth noting that /usr/bin/env python3 doesn't merely echo back a path to bash. If you type /usr/bin/env python3 directly on the command line, you'll see that it actually starts python (most likely via some flavor of exec()). So presumably your script would have to do the same when called. Commented Feb 27, 2019 at 21:58
  • Make your use case general. I believe it should be on the user to understand where their python path is. Commented Feb 27, 2019 at 22:04
  • read your execve man page: you can't put a script as the shebang. Commented Feb 28, 2019 at 1:04
  • I would suggest one tactic is to tell your users that python3 in the PATH is a requirement to run your program. Commented Feb 28, 2019 at 1:05

2 Answers 2

Reset to default
1

is it possible to replace the #!/usr/bin/env python3 at the top of the Python file with a call to a bash shell script that looks for the python location and then "sets" it for the script?

Yes, of course. That's effectively what /usr/bin/env python3 does. There's nothing magic about that particular command; it and variations on it just happen to be broadly useful.

I created a bash script that does look through the possible locations until it finds the interpreter, but what I don't know how to do is return it in the top of a Python file.

You have a misunderstanding about what's happening. A shebang line does not result in a substitution into the script. Rather, it results in the specified line being executed as a command, with the path to the original script and the arguments to it appended as an additional arguments.

Thus, your pyshebang.sh should have a general form along these lines:

#!/bin/bash # Note: the above shebang line is not a special case # ... find Python ... MY_PYTHON=/the/python/I/discovered # Execute the discovered Python, passing it all the arguments this # script received exec "$MY_PYTHON" "$@"
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Great. Thanks for the thorough explanation. I just tried it out and it worked exactly as I was intending. Thanks again!
Do you know of any reason why this methodology wouldn't work if the calling script is a CGI file that is a gateway to a Flask server? That is, I'm running a Flask server on an old Apache that only runs CGI (no FCGI, much less WSGI). My app.cgi shebang line points to a shell script, which follows your method of picking a Python then execing that python. However, Apache gives an error with logs such as: (8)Exec format error: exec of '.../my-flask-app/app.cgi' failed Premature end of script headers: app.cgi The redirection script works fine on command-line python scripts
No, @joelion, I am not aware of any reason why this general methodology would fail for you. At the level of detail presented, it relies only on well-known, well-documented features of Bash and an underlying POSIX-like operating system. I can think of a variety of details that, if implemented incorrectly, might possibly elicit behavior such as you describe, but that would probably be best approached in a separate question. If you pose one, then please be sure to include a minimal reproducible example.
0

You can use a hack like the following to have the shell find the correct interpreter, then re-run the current script using that interpreter.

#!/bin/sh ''':' cmd=$(./pyshebang) exec "$cmd" "$0" "$@" :' ''' print("Hello")

This is a shell script. The first command is :, constructed from an empty single-quoted string and a quoted :. : is a no-op. The next command sets the variable cmd to the path returned by ./pyshebang, followed by a command to replace the current process with the program indicated by $cmd, with the current script name as its first argument and taking any other arguments that were passed to the shell script.

When this same file is then run by a Python interpreter, it sees a doc-string ('''...''') whose contents (the shell code previously executed) are ignored. Execution then begins with the proper Python code which the shell never looked at before finding the exec command.

Improve this answer

2 Comments

I don't think it's right to pass $0 as an argument to the exec. Are you sure about that? I'm pretty sure the interpreter receives the target script name as an argument, not as its own name, so the interpreter shim the OP wants would just repeat its own arguments to the discovered Python command.
I'm assuming the shim is just returning something like /usr/bin/python; that's what gets execed, and the first argument it will expect is the script to actually run.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.