6

Edit:

Important note before reading further: since Kusalananda's anwser, my question is now quite useless, since he pointed out a bug in another script I used. Now this bug is fixed, the below errors I described no longer occur!

There are two issues in your code which causes the value in $extension to be expanded
The second issue is the function itself

TL;DR

I was stuck when I started writing the question, but found a solution that works (using shell noglob), BUT I still have a question of "would it be possible to write".

I'm sure this has already been answered, but I get lost in hundreds of topics about bash expansion.

Here is my case:

  • I want to run a script (lets say ffmpeg with option -pattern_type glob -i mode that accepts a regexp like *.JPG)
  • taking an argument (lets say *.JPG)
  • argument coming from a variable (lets say $extension)
  • variable coming from script parameters (lets say $1)
  • but I do not want shell to expand $extension as by default for a pattern like *.JPG

Note that all this actions take place in a script script.sh.

Thus, the user could run:

script.sh 'JPG' ; # where $1='JPG'

Note: in script.sh, I concatenate *. and $1 to get a more final regexp like *.JPG

I read (from here https://stackoverflow.com/a/11456496/912046) that shell expansion occurs before ffmpeg receives arguments (ffmpeg even does not know Shell expansion took place)

I tried to play with quotes, double quotes, or backslashes around ${extension} in different ways, unsuccessfully.
Here are some tries:

ffmpeg ... "${extension}"

leads to: ffmpeg 1.jpg 2.jpg [...] (expansion occurs)

ffmpeg ... ${extension}

same

ffmpeg ... '${extension}'

leads to no file match because of searching pattern ${extension} (variable name is used literally)

extension="\'${extension}\'" ; # wrapping the single quotes (preventing expansion) directly in variable ffmpeg ... ${extension}

leads to no file match because of searching pattern '*.jpg' (including single quotes)

ffmpeg ... \"${extension}\"

same but searching for "*.jpg" (including double quotes)

I finally succeeded using Shell noglob option.
Here is the command for curious:

set -f ; # disable glob (prevent expansion on '*.jpg') ffmpeg -pattern_type glob -i ${extension} movie.mp4 ; set +f ; # restore, but what if glob was already disabled? I should not restore it then?

But for my curiosity, is there another way of preventing expansion on a script argument coming from a variable we WANT to be resolved? (thus, the way I know of preventing expansion (using single quotes) makes variable no longer resolved/replaced).

Something like : (but I read string concatenation should be avoided for safety, injection)

ffmpeg '${extension}' || | 3 || 2 | 1

where :

  • 1: would open single quote for final result being ffmpeg '*.jpg'
  • 2: would inject and resolve our variable extension to .jpg
  • 3: would close single quote for ffmpeg to receive argument as if I manually had written '*.jpg'

Edit: after comment about how extension variable is assigned:

local extension="${2}" ; echo $extension ; extension="${extension:=jpg}" ; # Default on "jpg" extension # wrapp extension type in single quotes + prefix with "*." extension="*.${extension}" ;

Then, using it with:

runprintcommand ffmpeg -loglevel verbose -pattern_type glob -i ${extension} "$movieName" ;

Where the runprintcommand function/script being:

function runprintcommand() { echo "Command to run: (Note: '\\' escape character may not be printed)" ; echo "$*" ; $* ; }
Improve this question
3
  • 2
    "$extension" (or "${extension}") would expand the variable, but would under no circumstances perform filename globbing on the result, unless you used eval on the result of the expansion. You are doing something very strange if it did that. Or, ffmpeg might possibly do its own globbing, which it would do regardless of set -f in that case. Show us how you assign to extension to create it from $1. Commented Mar 6, 2019 at 21:42
  • indeed, I delegate the call of ffmpeg to a custom homemade, possibly broken runprintcommand function, which may interfer with globbing. Commented Mar 6, 2019 at 21:51
  • 2
    Then you have a bug in that function, or in the creation of the extension variable's value. Commented Mar 6, 2019 at 21:52

3 Answers 3

Reset to default
10
  • An unquoted variable would undergo variable expansion, word-splitting (on spaces, tabs and newlines by default), and each generated word would in turn undergo filename generation (globbing).
  • A variable within double quotes would have its value expanded, but the shell would not do word-splitting or globbing on the expanded value.
  • A variable within single quotes would not be expanded at all.

Example:

$ ls script.sh 
$ var='* *' 
$ echo $var script.sh script.sh 
$ echo "$var" * * 
$ echo '$var' $var

There are two issues in your code which causes the value in $extension to be expanded to the glob pattern that it contains and also causes the glob pattern to be matched against filenames prematurely (you want to hand it as-is to ffmpeg -i which does its own glob expansion internally).

The first is your call to your function:

runprintcommand ffmpeg -loglevel verbose -pattern_type glob -i ${extension} "$movieName" ;

Here, ${extension} is unquoted, so you would definitely get (word-splitting and) filename globbing happening on its value.

The second issue is the function itself:

function runprintcommand() { echo "Command to run: (Note: '\\' escape character may not be printed)" ; echo "$*" ; $* ; }

Here, you use $* unquoted, which again would (split the value up into words and then) expand the glob, even if you double-quoted ${extension} in the call to the function.

Instead of a bare $*, use "$@" (with the double quotes). This would expand to the individually quoted positional parameters of the function.

This is the difference between "$@" and "$*":

  • "$*" is a single double-quoted string. This can generally not be used to execute a command with arguments.
  • "$@" is a list of double quoted words. This could be used to execute a command with arguments.
Improve this answer
2
  • Originally, the unquoted ${extension} was a (single quoted, but hardcoded) '*.jpg' that I wanted to turn into an argument, for script to have such an option. (I want to say, its a remaining bug after my today tests, it has to be quoted I agree.). Thank you for the hint in runprintcommand ("$@") and for your explanations and your time. Commented Mar 6, 2019 at 22:07
  • 1
    For information, (you were right about the bug on other function), I replaced it with "$@" and the call by runprintcommand ffmpeg -loglevel verbose -pattern_type glob -i "${extension}" "$movieName" and it works perfectly now, variable is expanded, and word splitting and globbing NO longer occur. ffmpeg receives the expected *.JPG argument, and make its own globbing as @ikkachu does point out. Thanx again Commented Mar 6, 2019 at 22:27
2

The way to stop the shell from globbing is to quote the variable expansion. (It also stops word splitting, which is usually also what you want). That doesn't stop the command itself from processing glob-like patterns, of course. ffmpeg does that for -i, and so does e.g. find -name. Using the latter as an example:

$ touch a.foo b.foo c.bar $ extension=foo $ set -x $ find . -name "*.$extension" + find . -name '*.foo' ./b.foo ./a.foo

(The line that starts with a + comes from set -x, and shows the command the shell actually ran.)

While you can stop globbing with set -f, too, it doesn't help with word splitting. If the variable contained whitespace, the unquoted expansion would break, even with set -f.

Improve this answer
1
  • thank you for explanation, I fixed the bug that occured in fact in another script where I missed to double quote the right thing. Now that it is double quoted well at all the place it needs, no word splitting nor globbing will occur as expected. Commented Mar 6, 2019 at 22:30
1

As I found an alternate syntax of doing this, I write it down here.

By using a flag to disable Shell globbing (http://tldp.org/LDP/abs/html/globbingref.html), it does prevent expansion:

set -f ; # disable glob (prevent expansion on '*.jpg' to turn in "1.jpg 2.jpg 3.jpg...") ffmpeg -pattern_type glob -i ${extension} movie.mp4 ; set +f ; # restore
Improve this answer
1
  • That's what I was looking for. Easy. Commented Feb 19, 2022 at 18:25

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.