command runs in terminal but not via /bin/sh

Question

If I run this command it works fine in the terminal:

for dirname in $(ls -d dir/checkpoint/features.txt/20*);do;echo "hello";done

But when run through /bin/sh -c it gives an error

/bin/sh -c "for dirname in $(ls -d dir/checkpoint/features.txt/20*);do;echo "hello";done"

ERROR:

/bin/sh: -c: line 1: syntax error near unexpected token `dir/checkpoint/features.txt/201108000' /bin/sh: -c: line 1: `dir/checkpoint/features.txt/201108000'

My default shell is /bin/bash. I cant seem to understand what is causing this. My default implementation for running all shell commands in my program is by appending /bin/sh -c to them. It is the first time i am seeing this issue. Any suggestions?

Is /bin/sh the same as /bin/bash? On (older versions of) Solaris, for instance, /bin/sh is not a POSIX shell and does not recognize $(...) notation. Did you try enclosing the command in single quotes. — Jonathan Leffler
– Jonathan Leffler, Commented Feb 1, 2013 at 0:20
Since you've added another set of double quotes, sh is not parsing it the way you think it reads. Use single quotes on the outside instead. — chris
– chris, Commented Feb 1, 2013 at 0:22
@JonathanLeffler This shows the difference between the two shells askubuntu.com/questions/141928/…. I tried with single quotes too but it disnt work — shaurya
– shaurya, Commented Feb 1, 2013 at 0:25

bta · Accepted Answer · 2013-02-01 23:02:25Z

Don't try to parse the output of ls, especially with a for construct. There are many, many ways that this can go wrong.

This is a good place to use find instead. Try this:

/bin/sh -c "find dir/checkpoint/features.txt -mindepth 1 -maxdepth 1 -type d -iname '20*' -exec echo \"hello\" \;"

Besides eliminating the error-prone use of ls, you avoid the sub-shell and all of the issues that it brings with it.

Follow-up in response to your comment:

I'm assuming that you're using awk -F/ '{print $NF}' to grab the name of the folder in which the file lives (that is, the last directory name before the filename). The commands basename and dirname can be used to do this for you. This should make your script a bit easier. Place the following into a script file:

#!/bin/sh folder=$(basename $(dirname $1)) mkdir -p #{nfs_checkpoint}/${folder} cat #{result_location}/${folder}/20* > #{nfs_checkpoint}/${folder}/features.txt

And execute it like this:

/bin/sh -c "find dir/checkpoint/features.txt -mindepth 1 -maxdepth 1 -type d -iname '20*' -exec yourscript.sh {} \;"

It would be clearer to switch to single quotes on the outside instead of trying to escape the inner double quotes. /bin/sh -c 'foo "bar"' instead of /bin/sh -c "foo \"bar\"".
Thanks for the answer! Well just for the sake of simplicity I pasted the command like that in the question but the real one is for dirname in $(ls -d #{result_location}/20*|awk -F/ '{print $NF}');do;mkdir -p #{nfs_checkpoint}/${dirname};cat #{result_location}/${dirname}/20* > #{nfs_checkpoint}/${dirname}/features.txt;done. The thing i am struggling with mainly is how to get just the filename (not filepath) through find
ls -b might also work. At least for the GNU user land. However, I also prefer find and pipe that into a while read ...
@0xC0000022L- No, for still chokes on the output of ls -b if a file or folder name has spaces in it. find is really the only way to do this safely.

Collectives™ on Stack Overflow

command runs in terminal but not via /bin/sh

1 Answer 1

4 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Related