I want to ask the shell to display a csv file in a nice format in my python script. So I wrote the following:
printout = "column -s, -t < output.csv | less -#2 -N -S " subprocess.call(printout.split(), shell = False) The error I get is:
column: invalid option -- '#' I have a rough idea that it is something to do with shell=False; however when I set it to True and run in cmd line, it puts me into another line and I have to ctrl+C to get out.
Your original code was equivalent to the shell command:
# this can be used to reproduce your bug at a shell column -s, -t '<' output.csv '|' less -#2 -N -S ...passing <, |, less, etc. as arguments to column, not treating them as shell directives.
See the section Replacing Shell Pipelines in the subprocess module documentation.
p1 = subprocess.Popen(['column', '-s,', '-t'], stdin=open('output.csv', 'r'), stdout=subprocess.PIPE) p2 = subprocess.Popen(['less', '-#2', '-N', '-S'], stdin=p1.stdout) p1.stdout.close() # ensure that p2 has the only remaining handle on p1.stdout p2.communicate() # let less run 
split() or even shlex.split() can be intentionally misled or broken by unexpected or malicious filenames -- think about what happens if your output.csv is from a variable and the filename has a space in it.
"'%s'" % filename with shlex.split() or shell=True, but even that doesn't work if the filename contains literal 's that can then escape the syntactical ones.execve syscall, and it takes a list of C strings. Anything you do with a single string instead of a list is going to need to be broken down / transformed into that list before it can actually be used; passing a list yourself means you have absolutely control, instead of relying on whatever does that transformation to do it the way you intend.
shell=Falsemeans you don't have a shell, so there's nothing that knows how to process redirections (<) or pipes (|). It's actually better to useshell=False, but it means you need to do more of your setup in Python.