I am trying to capture the output of a process I launch through python for logging purposes. But for some reason despite all my attempts none of the solutions seems to be working for this particular case. Can someone help? My code looks like this:
cmd = 'cat input.fq | fastx_clipper -Q33 -a TGGAATTCTCGGGTGCC -l 5 -c -n -v | fastx_trimmer -Q33 -f 2 > ../results/trimmed.fq' p = Popen(cmd,shell=True,stdout = PIPE) for line in p.stdout:output.write(line) > ... redirects output of the command to a file; you will get no output from the command because of that. Remove > ... part:
cmd = 'cat input.fq | fastx_clipper -Q33 -a TGGAATTCTCGGGTGCC -l 5 -c -n -v | fastx_trimmer -Q33 -f 2' > ../results/trimmed.fq will indeed redirect the output of your command into ../results/trimmed.fq as falsetru said, so you shouldn't expect to have any lines to print in p.stdout. If however that file is still empty after running the command, I would suspect that there is something wrong with input.fq or the arguments into fastx_clipper.
Also I just want to point out that if you want to both write output to a file and display it in terminal, you can use the tee command. It would look like this:
'cat input.fq | fastx_clipper -Q33 -a TGGAATTCTCGGGTGCC -l 5 -c -n -v | fastx_trimmer -Q33 -f 2 | tee ../results/trimmed.fq' 
../results/trimmed.fqwith your shell command, so why would you be expecting any output from stdout?> ../results/trimmed.fq. If you run that command in a terminal, do you see any output? Or is everything being dumped to that file?