4

Explaining the question through examples...

Demonstrates that the single-quotes after --chapters is gets escaped when the variable is expanded (I didn't expect this):

prompt@ubuntu:/my/scripts$ cat test1.sh #!/bin/bash actions="--tags all:" actions+=" --chapters ''" mkvpropedit "$1" $actions prompt@ubuntu:/my/scripts$ ./test1.sh some.mkv Error: Could not open '''' for reading.

And now for some reason mkvpropedit receives the double quotes as part of the filename (I didn't expect this either): 

prompt@ubuntu:/my/scripts$ cat test1x.sh #!/bin/bash command="mkvpropedit \"$1\"" command+=" --tags all:" command+=" --chapters ''" echo "$command" $command prompt@ubuntu:/my/scripts$ ./test1x.sh some.mkv mkvpropedit "some.mkv" --tags all: --chapters '' Error: Could not open '''' for reading.

The above echo'd command seems to be correct. Putting the same text in another script gives the expected result:

prompt@ubuntu:/my/scripts$ cat test2.sh #!/bin/bash mkvpropedit "$1" --tags all: --chapters '' prompt@ubuntu:/my/scripts$ ./test2.sh some.mkv The file is being analyzed. The changes are written to the file. Done.

Could anyone please explain why the quotes are not behaving as expected. I found searching on this issue difficult as there are so many other quoting discussions on the web. I wouldn't even know how to explain the question without examples.

I am afraid that some day the file name in the argument contains some character that breaks everything, hence the maybe excessive quoting. I do not understand why the same command executes differently when typed directly in the script or when provided via a variable. Please enlighten me.

Thanks for reading.

Improve this question
0

1 Answer 1

Reset to default
5

The important thing to keep in mind is that quotes are only removed once, when the command line is originally parsed. A quote which is inserted into the command line as a result of parameter substitution ($foo) or command substitution ($(cmd args)) is not treated as a special character. [Note 1]

That seems different from whitespace and glob metacharacters. Word splitting and pathname expansion happen after parameter/command substitution (unless the substitution occurs inside quotes). [Note 2]

The consequence is that it is almost impossible to create a bash variable $args such that

cmd $args

If $args contains quotes, they are not removed. Words inside $args are delimited by sequences of whitespace, not single whitespace characters.

The only way to do it is to set $IFS to include some non-whitespace character; that character can then be used inside $args as a single-character delimiter. However, there is no way to quote a character inside a value, so once you do that, the character you chose cannot be used other than as a delimiter. This is not usually very satisfactory.

There is a solution, though: bash arrays.

If you make $args into an array variable, then you can expand it with the repeated-quote syntax:

cmd "${args[@]}"

which produces exactly one word per element of $args, and suppresses word-splitting and pathname expansion on those words, so they end up as literals.

So, for example:

actions=(--tags all:) actions+=(--chapters '') mkvpropedit "$1" "${actions[@]}"

will probably do what you want. So would:

args=("$1") args+=(--tags) args+=(all:) args+=(--chapters) args+=('') mkvpropedit "${args[@]}"

and so would

command=(mkvpropedit "$1" --tags all: --chapters '') "${command[@]}"

I hope that's semi-clear.

man bash (or the online version) contains a blow-by-blow account of how bash assembles commands, starting at the section "EXPANSION". It's worth reading for a full explanation.

Notes:

  1. This doesn't apply to eval or commands like bash -c which evaluate their argument again after command line processing. But that's because command-line processing happens twice.

  2. Word splitting is not the same as "dividing the command into words", which happens when the command is parsed. For one thing, word-splitting uses as separator characters the value of $IFS, whereas command-line parsing uses whitespace. But neither of these are done inside quotes, so they are similar in that respect. In any case, words are split in one way or another both before and after parameter substitution.

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

hi @rici, thanks for your quick and most helpful answer. I played a bit with both approaches (eval and arrays), but so far have not been able to decide which I like more. Both do the trick, so I am happy, even more so because I now finally really understand the behaviour. Kudos
@PaulvanLeeuwen: Avoid eval when at all possible. Since it applies another full pass of command-line processing, it can wind up parsing more than you wanted. For example, suppose $1 is Joe's movie.mkv -- it'll take the apostrophe as a quote mark, fail to find a matching close quote, and error out. Or suppose $1 is mistake $(rm *).mkv -- it'll actually execute the command rm * as part of evaluating the command string. Arrays are a far far better way to do this. BTW, please read BashFAQ #050 and #048.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.