In a shell script, how do I echo all shell commands called and expand any variable names?
For example, given the following line:
ls $DIRNAME I would like the script to run the command and display the following
ls /full/path/to/some/dir The purpose is to save a log of all shell commands called and their arguments. Is there perhaps a better way of generating such a log?
set -x or set -o xtrace expands variables and prints a little + sign before the line.
set -v or set -o verbose does not expand the variables before printing.
Use set +x and set +v to turn off the above settings.
On the first line of the script, one can put #!/bin/sh -x (or -v) to have the same effect as set -x (or -v) later in the script.
The above also works with /bin/sh.
See the bash-hackers' wiki on set attributes, and on debugging.
$ cat shl #!/bin/bash DIR=/tmp/so ls $DIR $ bash -x shl + DIR=/tmp/so + ls /tmp/so $ 
bash -x foo.sh was the crux of what I needed. (Posting as a comment because it wasn't immediately apparent whether that would work without modifying the script itself; it did. )set -x will give you what you want.
Here is an example shell script to demonstrate:
#!/bin/bash set -x #echo on ls $PWD This expands all variables and prints the full commands before output of the command.
Output:
+ ls /home/user/ file1.txt file2.txt 
set -o verbose or set -v (only "verbose") or set -o xtrace or set -x (only "xtrace") or set -xv (both) or set -o xtrace -o verbose (both).
I use a function to echo and run the command:
#!/bin/bash # Function to display commands exe() { echo "\$ $@" ; "$@" ; } exe echo hello world Which outputs
$ echo hello world hello world For more complicated commands pipes, etc., you can use eval:
#!/bin/bash # Function to display commands exe() { echo "\$ ${@/eval/}" ; "$@" ; } exe eval "echo 'Hello, World!' | cut -d ' ' -f1" Which outputs is this
$ echo 'Hello, World!' | cut -d ' ' -f1 Hello, 
++ set +x output when turned off, as well as looking cleaner. For just a single statement or two, though, bhassel's answer using a subshell is the most convenient.
cp "foo bar" baz and cp foo "bar baz", for example. So it's good for displaying progress information to a user; less so for debugging output or recording reproducible commands. Different use cases. In zsh, you can preserve quoting with the :q modifier: exe() { echo '$' "${@:q}" ; "$@" ; }
exe (cut -d ' ' -f9,10 --complement ${folder}/hog_crop_px.txt) | paste - ${folder}/lbp_crop_px.txt > ${folder}/crop_px.tx gives me syntax error near unexpected token cut'` even though it executes perfectly without the exe() command. Any suggestion?
eval from your command. So don't expect it to work properly on exe eval "echo 'eval world'"!You can also toggle this for select lines in your script by wrapping them in set -x and set +x, for example,
#!/bin/bash ... if [[ ! -e $OUT_FILE ]]; then echo "grabbing $URL" set -x curl --fail --noproxy $SERV -s -S $URL -o $OUT_FILE set +x fi 
set +x too
{ set +x; } &> /dev/null to silence the set +x output.
shuckc's answer for echoing select lines has a few downsides: you end up with the following set +x command being echoed as well, and you lose the ability to test the exit code with $? since it gets overwritten by the set +x.
Another option is to run the command in a subshell:
echo "getting URL..." ( set -x ; curl -s --fail $URL -o $OUTFILE ) if [ $? -eq 0 ] ; then echo "curl failed" exit 1 fi which will give you output like:
getting URL... + curl -s --fail http://example.com/missing -o /tmp/example curl failed This does incur the overhead of creating a new subshell for the command, though.
++ set +x output.
if [ $? -eq 0 ] with if (set -x; COMMAND).
[ $? -eq 0 ]? Or this is mistake? Zero exit code means success. May be you mean [ $? -nq 0 ]; Also better to return non hardcoded 1 exit code, but the one which returned by $?.According to TLDP's Bash Guide for Beginners: Chapter 2. Writing and debugging scripts:
2.3.1. Debugging on the entire script
$ bash -x script1.sh...
There is now a full-fledged debugger for Bash, available at SourceForge. These debugging features are available in most modern versions of Bash, starting from 3.x.
2.3.2. Debugging on part(s) of the script
set -x # Activate debugging from here w set +x # Stop debugging from here...
Table 2-1. Overview of set debugging options
Short | Long notation | Result -------+---------------+-------------------------------------------------------------- set -f | set -o noglob | Disable file name generation using metacharacters (globbing). set -v | set -o verbose| Prints shell input lines as they are read. set -x | set -o xtrace | Print command traces before executing command. ...
Alternatively, these modes can be specified in the script itself, by adding the desired options to the first line shell declaration. Options can be combined, as is usually the case with UNIX commands:
#!/bin/bash -xv
You can execute a Bash script in debug mode with the -x option.
This will echo all the commands.
bash -x example_script.sh # Console output + cd /home/user + mv text.txt mytext.txt You can also save the -x option in the script. Just specify the -x option in the shebang.
######## example_script.sh ################### #!/bin/bash -x cd /home/user mv text.txt mytext.txt ############################################## ./example_script.sh # Console output + cd /home/user + mv text.txt mytext.txt 
bash -vx will do the same but without variable interpolation
Another option is to put "-x" at the top of your script instead of on the command line:
$ cat ./server #!/bin/bash -x ssh user@server $ ./server + ssh user@server user@server's password: ^C $ 
./myScript and bash myScript. Still a good thing to point out, thanks.Combining all the answers I found this to be the best, simplest
#!/usr/bin/env bash # https://stackoverflow.com/a/64644990/8608146 exe(){ set -x "$@" { set +x; } 2>/dev/null } # example exe go generate ./... { set +x; } 2>/dev/null from https://stackoverflow.com/a/19226038/8608146
If the exit status of the command is needed, as mentioned here
Use
{ STATUS=$?; set +x; } 2>/dev/null And use the $STATUS later like exit $STATUS at the end
A slightly more useful one
#!/usr/bin/env bash # https://stackoverflow.com/a/64644990/8608146 _exe(){ [ $1 == on ] && { set -x; return; } 2>/dev/null [ $1 == off ] && { set +x; return; } 2>/dev/null echo + "$@" "$@" } exe(){ { _exe "$@"; } 2>/dev/null } # examples exe on # turn on same as set -x echo This command prints with + echo This too prints with + exe off # same as set +x echo This does not # can also be used for individual commands exe echo what up! 
#!/bin/bash to the top of both scripts.Type "bash -x" on the command line before the name of the Bash script. For instance, to execute foo.sh, type:
bash -x foo.sh For zsh, echo
setopt VERBOSE And for debugging,
setopt XTRACE 
verbose can be in lowercase. To remove it, use unsetopt instead of setopt. Example: unsetopt verbose. PS: This doesn't work in scripts though.To allow for compound commands to be echoed, I use eval plus Soth's exe function to echo and run the command. This is useful for piped commands that would otherwise only show none or just the initial part of the piped command.
Without eval:
exe() { echo "\$ $@" ; "$@" ; } exe ls -F | grep *.txt Outputs:
$ file.txt With eval:
exe() { echo "\$ $@" ; "$@" ; } exe eval 'ls -F | grep *.txt' Which outputs
$ exe eval 'ls -F | grep *.txt' file.txt Following the answer of Soth.
It is possible to create a markdown output without highlight (no language given)
set -x exe() { echo "\`\$\$ ${@/eval/} \`" ; "$@" ; } set -x exe() { echo "\`\$\$ ${@/eval/} \`" ; "$@" ; } echo echo ----------------------------------------------------------- echo # Setup echo Lets take a random keyframe from in.mp4: echo exe eval "kfn=20" echo echo "kf=\$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | grep 'frame,video,0,1' | head -$kfn | tail -1 | perl -pe 's|frame,video,0,1,.*?,(.*?),.*|\1|') " exe eval "kf=$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | grep 'frame,video,0,1' | head -$kfn | tail -1 | perl -pe 's|frame,video,0,1,.*?,(.*?),.*|\1|') " echo echo Lets select keyframe at $kf. Here are the timestamps of the all the frames from in.mp4 around this keyframe. echo exe eval "ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2 \1|' | perl -pe 's|(.*?) 1|\1\tKF|' | perl -pe 's|(.*?) 0|\1|' |grep -A 5 -B 5 --color $kf" echo echo Lets compare 2 methods of split: actual losslesscut 3.53.0 and another one echo Lets take a random keyframe from in.mp4:
$$ kfn=20
kf=$(ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | grep 'frame,video,0,1' | head -20 | tail -1 | perl -pe 's|frame,video,0,1,.?,(.?),.*|\1|')
$$ kf=3.803792
Lets select keyframe at 3.803792. Here are the timestamps of the all the frames from in.mp4 around this keyframe.
$$ ffprobe -v error -select_streams v -show_frames -print_format csv in.mp4 | perl -pe 's|frame,video,0,(.*?),.*?,(.*?),.*|\2 \1|' | perl -pe 's|(.*?) 1|\1\tKF|' | perl -pe 's|(.*?) 0|\1|' |grep -A 5 -B 5 --color 3.803792
3.720375 3.737083 3.753750 3.770417 3.787125 **3.803792** KF 3.820500 3.837167 3.853833 3.870542 3.887208 If you want to log only specific commands, rather than every single line, set -x is somewhat difficult to use. But you can use this function to print any command, prefixed with $ :
function run() { echo -n '$' # Print $ without a newline. for arg in "$@"; do printf " %q" "$arg" # Print $arg, properly escaped. done echo # Print a newline. "$@" } Usage:
run ls /full/path/to/some/dir Using printf with the %q format specifier makes sure that the output is printed in a way suitable for copying and pasting into a shell, even in the presence of spaces or other special characters. It seems to escape spaces using \ rather than adding quotes, but the output does correctly represent what's actually being run.
I use the following to control exactly what I want to see:
#!/bin/bash cmd="ls $PWD" echo $cmd; eval $cmd cmd="cat $PWD/whatfille.txt" echo "-> $cmd"; eval $cmd #this prefixes to command for better reading The key to expand a variable is to use double quote. Single quote would not expand.
I sometimes use "logger" instead of "echo" to send to /var/log/syslog since they are often run non-interactively.
For csh and tcsh, you can set verbose or set echo (or you can even set both, but it may result in some duplication most of the time).
The verbose option prints pretty much the exact shell expression that you type.
The echo option is more indicative of what will be executed through spawning.
http://www.tcsh.org/tcsh.html/Special_shell_variables.html#verbose
http://www.tcsh.org/tcsh.html/Special_shell_variables.html#echo
Special shell variables
verbose If set, causes the words of each command to be printed, after history substitution (if any). Set by the -v command line option.
echo If set, each command with its arguments is echoed just before it is executed. For non-builtin commands all expansions occur before echoing. Builtin commands are echoed before command and filename substitution, because these substitutions are then done selectively. Set by the -x command line option.
$ cat exampleScript.sh #!/bin/bash name="karthik"; echo $name; bash -x exampleScript.sh Output is as follows:
xargs will still execute its command once when there is no input. It also provides the -t option to output the command before executing it. For example:
% xargs -a /dev/null -t echo foo echo foo foo % This also works in a pipe. For example:
% echo foo | xargs -a /dev/null -t wc -m wc -m 4 % This can also be used in Makefile recipes when you want to suppress displaying shell expressions but still display the resulting commands without duplicating them with echo or printf. For example:
X := xargs -a /dev/null -t clean: @if [ -f $(BUILD_DIR)/secrets.h ]; then \ $(X) rm -f $(BUILD_DIR)/secrets.h; \ fi % make clean rm -f build/secrets.h % To answer the original question regarding ls, using bash you could do this:
% alias X='xargs -a /dev/null -t' % DIRNAME=/usr/games % X ls $DIRNAME ls /usr/games gamemodelist gamemoderun gamemode-simulate-game % 