205

I have a variable in my bash script whose value is something like this:

~/a/b/c

Note that it is unexpanded tilde. When I do ls -lt on this variable (call it $VAR), I get no such directory. I want to let bash interpret/expand this variable without executing it. In other words, I want bash to run eval but not run the evaluated command. Is this possible in bash?

How did I manage to pass this into my script without expansion? I passed the argument in surrounding it with double quotes.

Try this command to see what I mean:

ls -lt "~"

This is exactly the situation I am in. I want the tilde to be expanded. In other words, what should I replace magic with to make these two commands identical:

ls -lt ~/abc/def/ghi

and

ls -lt $(magic "~/abc/def/ghi")

Note that ~/abc/def/ghi may or may not exist.

Improve this question
3
  • 6
    You might find Tilde expansion in quotes helpful too. It mostly, but not entirely, avoids using eval. Commented Dec 28, 2014 at 9:21
  • 6
    How did your variable get assigned with an unexpanded tilde? Maybe all that is required is assign that variable with the tilde outside quotes. foo=~/"$filepath" or foo="$HOME/$filepath" Commented Jan 6, 2017 at 23:45
  • dir="$(readlink -f "$dir")" Commented Feb 13, 2020 at 11:51

19 Answers 19

Reset to default
186

If the variable var is input by the user, eval should not be used to expand the tilde using

eval var=$var # Do not use this!

The reason is: the user could by accident (or by purpose) type for example var="$(rm -rf $HOME/)" with possible disastrous consequences.

A better (and safer) way is to use Bash parameter expansion:

var="${var/#\~/$HOME}"
Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

How could you change ~userName/ instead of just ~/ ?
@aspergillusOryzae Good question. Here is a workaround: stackoverflow.com/a/2069835/2173773
What is the purpose of # in "${var/#\~/$HOME}" ?
@Jahid It is explained in the manual . It forces the tilde to only match at the beginning of $var.
Please edit into your answer the explanation that "${var/#...}" is a special bash syntax to only match at the beginning. With URL. I've used bash for decades but didn't know that one. Also that the possible problem with eval() is malicious code injection.
|
118

Due to the nature of StackOverflow, I can't just make this answer unaccepted, but in the intervening 5 years since I posted this there have been far better answers than my admittedly rudimentary and pretty bad answer (I was young, don't kill me).

The other solutions in this thread are safer and better solutions. Preferably, I'd go with either of these two:

Original answer for historic purposes (but please don't use this)

If I'm not mistaken, "~" will not be expanded by a bash script in that manner because it is treated as a literal string "~". You can force expansion via eval like this.

#!/bin/bash homedir=~ eval homedir=$homedir echo $homedir # prints home path

Alternatively, just use ${HOME} if you want the user's home directory.

Improve this answer

17 Comments

Do you have a fix for when the variable has a space in it?
I found ${HOME} most attractive. Is there any reason not to make this your primary recommendation? In any case, thanks!
@sage No, I'd prefer ${HOME} as well - but my answer was explaining how to do it with ~ like the question was asking.
+1 -- I was needing to expand ~$some_other_user and eval works fine when $HOME will not work because I don't need the current user home.
Using eval is a horrible suggestion, it's really bad that it gets so many upvotes. You will run into all sorts of problems when the variable's value contains shell meta characters.
|
31

Plagarizing myself from a prior answer, to do this robustly without the security risks associated with eval:

expandPath() { local path local -a pathElements resultPathElements IFS=':' read -r -a pathElements <<<"$1" : "${pathElements[@]}" for path in "${pathElements[@]}"; do : "$path" case $path in "~+"/*) path=$PWD/${path#"~+/"} ;; "~-"/*) path=$OLDPWD/${path#"~-/"} ;; "~"/*) path=$HOME/${path#"~/"} ;; "~"*) username=${path%%/*} username=${username#"~"} IFS=: read -r _ _ _ _ _ homedir _ < <(getent passwd "$username") if [[ $path = */* ]]; then path=${homedir}/${path#*/} else path=$homedir fi ;; esac resultPathElements+=( "$path" ) done local result printf -v result '%s:' "${resultPathElements[@]}" printf '%s\n' "${result%:}" }

...used as...

path=$(expandPath '~/hello')

Alternately, a simpler approach that uses eval carefully:

expandPath() { case $1 in ~[+-]*) local content content_q printf -v content_q '%q' "${1:2}" eval "content=${1:0:2}${content_q}" printf '%s\n' "$content" ;; ~*) local content content_q printf -v content_q '%q' "${1:1}" eval "content=~${content_q}" printf '%s\n' "$content" ;; *) printf '%s\n' "$1" ;; esac }
Improve this answer

26 Comments

Looking at your code it looks like you're using a cannon to kill a mosquito. There's got to be a much simpler way..
@Gino, there's surely a simpler way; the question is whether there's a simpler way that's also secure.
@Gino, ...I do suppose that one can use printf %q to escape everything but the tilde, and then use eval without risk.
@Gino, ...and so implemented.
Safe, yes, but very much incomplete. My code isn't complex for the fun of it -- it's complex because the actual operations done by tilde expansion are complex.
|
15

Here is a ridiculous solution:

$ echo "echo $var" | bash

An explanation of what this command does:

  1. create a new instance of bash, by... calling bash;
  2. take the string "echo $var" and substitute $var with the value of the variable (thus after the substitution the string will contain the tilde);
  3. take the string produced by step 2 and send it to the instance of bash created in step one, which we do here by calling echo and piping its output with the | character.

Basically the current bash instance we're running takes our place as the user of another bash instance and types in the command "echo ~..." for us.

Improve this answer

3 Comments

While your answer may solve the question, including an explanation of how and why this solves the problem would really help to improve the quality of your post, and probably result in more up-votes. Remember that you are answering the question for readers in the future, not just the person asking now. You can edit your answer to add explanations and give an indication of what limitations and assumptions apply. - From Review
Thanks, this helps to resolve a path like ~$USERNAME before passing it to realpath command e.g. realpath -qe `echo "echo ~$USERNAME/.." | bash`
Wouldn't that still allow a nefarious actor to run code?
12

How about this:

path=`realpath "$1"`

Or:

path=`readlink -f "$1"`
Improve this answer

9 Comments

looks nice, but realpath does not exist on my mac. And you would have to write path=$(realpath "$1")
Hi @Hugo. You can compile your own realpath command in C. For instance, you can generate an executable realpath.exe using bash and gcc from this command line: gcc -o realpath.exe -x c - <<< $'#include <stdlib.h> \n int main(int c,char**v){char p[9999]; realpath(v[1],p); puts(p);}'. Cheers
@Quuxplusone not true, at least on linux: realpath ~ -> /home/myhome
iv'e used it with brew on mac
@dangonfast This won't work if you set the tilde into quotes, the result is <workingdir>/~.
|
9

A safe way to use eval is "$(printf "~/%q" "$dangerous_path")". Note that is bash specific.

#!/bin/bash relativepath=a/b/c eval homedir="$(printf "~/%q" "$relativepath")" echo $homedir # prints home path

See this question for details

Also, note that under zsh this would be as as simple as echo ${~dangerous_path}

Improve this answer

3 Comments

echo ${~root} give me no output on zsh (mac os x)
export test="~root/a b"; echo ${~test}
That Zsh solution can have undesirable results if the file name contains any glob character.
7

Expanding (no pun intended) on birryree's and halloleo's answers: The general approach is to use eval, but it comes with some important caveats, namely spaces and output redirection (>) in the variable. The following seems to work for me:

mypath="$1" if [ -e "`eval echo ${mypath//>}`" ]; then echo "FOUND $mypath" else echo "$mypath NOT FOUND" fi

Try it with each of the following arguments:

'~' '~/existing_file' '~/existing file with spaces' '~/nonexistant_file' '~/nonexistant file with spaces' '~/string containing > redirection' '~/string containing > redirection > again and >> again'

Explanation

  • The ${mypath//>} strips out > characters which could clobber a file during the eval.
  • The eval echo ... is what does the actual tilde expansion
  • The double-quotes around the -e argument are for support of filenames with spaces.

Perhaps there's a more elegant solution, but this is what I was able to come up with.

Improve this answer

2 Comments

You might consider looking at behavior with names containing $(rm -rf .).
Doesn't this break on paths that actually contain > characters, though?
5

why not delve straight into getting the user's home directory with getent?

$ getent passwd mike | cut -d: -f6 /users/mike
Improve this answer

Comments

3

Here is the POSIX function equivalent of Håkon Hægland's Bash answer

expand_tilde() { tilde_less="${1#\~/}" [ "$1" != "$tilde_less" ] && tilde_less="$HOME/$tilde_less" printf '%s' "$tilde_less" }

2017-12-10 edit: add '%s' per @CharlesDuffy in the comments.

Improve this answer

1 Comment

printf '%s\n' "$tilde_less", perhaps? Otherwise it'll misbehave if the filename being expanded contain backslashes, %s, or other syntax meaningful to printf. Other than that, though, this is a great answer -- correct (when bash/ksh extensions don't need to be covered), obviously safe (no mucking with eval) and terse.
3

For anyone's reference, a function to mimic python's os.path.expanduser() behavior (no eval usage):

# _expand_homedir_tilde ~/.vim /root/.vim # _expand_homedir_tilde ~myuser/.vim /home/myuser/.vim # _expand_homedir_tilde ~nonexistent/.vim ~nonexistent/.vim # _expand_homedir_tilde /full/path /full/path

And the function:

function _expand_homedir_tilde { ( set -e set -u p="$1" if [[ "$p" =~ ^~ ]]; then u=`echo "$p" | sed 's|^~\([a-z0-9_-]*\)/.*|\1|'` if [ -z "$u" ]; then u=`whoami` fi h=$(set -o pipefail; getent passwd "$u" | cut -d: -f6) || exit 1 p=`echo "$p" | sed "s|^~[a-z0-9_-]*/|${h}/|"` fi echo $p ) || echo $1 }
Improve this answer

Comments

2

I believe this is what you're looking for 

magic() { # returns unexpanded tilde express on invalid user local _safe_path; printf -v _safe_path "%q" "$1" eval "ln -sf ${_safe_path#\\} /tmp/realpath.$$" readlink /tmp/realpath.$$ rm -f /tmp/realpath.$$ }

Example usage:

$ magic ~nobody/would/look/here /var/empty/would/look/here $ magic ~invalid/this/will/not/expand ~invalid/this/will/not/expand
Improve this answer

3 Comments

I'm surprised that printf %q doesn't escape leading tildes -- it's almost tempting to file this as a bug, as it's a situation in which it fails at its stated purpose. However, in the interim, a good call!
Actually -- this bug is fixed at some point between 3.2.57 and 4.3.18, so this code no longer works.
Good point, I've adjusted to code to remove the leading \ if it exists, so all fixed and worked :) I was testing without quoting the arguments, so it was expanding before calling the function.
2

Simplest: replace 'magic' with 'eval echo'.

$ eval echo "~" /whatever/the/f/the/home/directory/is

Problem: You're going to run into issues with other variables because eval is evil. For instance:

$ # home is /Users/Hacker$(s) $ s="echo SCARY COMMAND" $ eval echo $(eval echo "~") /Users/HackerSCARY COMMAND

Note that the issue of the injection doesn't happen on the first expansion. So if you were to simply replace magic with eval echo, you should be okay. But if you do echo $(eval echo ~), that would be susceptible to injection.

Similarly, if you do eval echo ~ instead of eval echo "~", that would count as twice expanded and therefore injection would be possible right away.

Improve this answer

1 Comment

Contrary to what you said, this code is unsafe. For example, test s='echo; EVIL_COMMAND'. (It will fail because EVIL_COMMAND doesn’t exist on your computer. But if that command had been rm -r ~ for example, it would have deleted your home directory.)
2

I have done this with variable parameter substitution after reading in the path using read -e (among others). So the user can tab-complete the path, and if the user enters a ~ path it gets sorted. 

read -rep "Enter a path: " -i "${testpath}" testpath testpath="${testpath/#~/${HOME}}" ls -al "${testpath}"

The added benefit is that if there is no tilde nothing happens to the variable, and if there is a tilde but not in the first position it is also ignored.

(I include the -i for read since I use this in a loop so the user can fix the path if there is a problem.)

Improve this answer

Comments

1

Here's my solution:

#!/bin/bash expandTilde() { local tilde_re='^(~[A-Za-z0-9_.-]*)(.*)' local path="$*" local pathSuffix= if [[ $path =~ $tilde_re ]] then # only use eval on the ~username portion ! path=$(eval echo ${BASH_REMATCH[1]}) pathSuffix=${BASH_REMATCH[2]} fi echo "${path}${pathSuffix}" } result=$(expandTilde "$1") echo "Result = $result"
Improve this answer

14 Comments

Also, relying on echo means that expandTilde -n isn't going to behave as expected, and behavior with filenames containing backslashes is undefined by POSIX. See pubs.opengroup.org/onlinepubs/009604599/utilities/echo.html
Good catch. I normally use a one-user machine so I didn't think to handle that case. But I think the function could easily be enhanced to handle this other case by grepping through the /etc/passwd file for the otheruser. I'll leave it as an exercise for someone else :).
I've already done that exercise (and handled the OLDPWD case and others) in an answer you deemed too complex. :)
actually, i just found a fairly simple one-line solution that should handle the otheruser case: path=$(eval echo $orgPath)
FYI: I just updated my solution so that it can now handle ~username correctly. And, it should be fairly safe as well. Even if you put in a '/tmp/$(rm -rf /*)' as an argument, it should handle it gracefully.
|
0

Just to extend birryree's answer for paths with spaces: You cannot use the eval command as is because it seperates evaluation by spaces. One solution is to replace spaces temporarily for the eval command:

mypath="~/a/b/c/Something With Spaces" expandedpath=${mypath// /_spc_} # replace spaces eval expandedpath=${expandedpath} # put spaces back expandedpath=${expandedpath//_spc_/ } echo "$expandedpath" # prints e.g. /Users/fred/a/b/c/Something With Spaces" ls -lt "$expandedpath" # outputs dir content

This example relies of course on the assumption that mypath never contains the char sequence "_spc_".

Improve this answer

1 Comment

Doesn't work with tabs, or newlines, or anything else in IFS... and doesn't provide security around metacharacters like paths containing $(rm -rf .)
0

Just use eval correctly: with validation.

case $1${1%%/*} in ([!~]*|"$1"?*[!-+_.[:alnum:]]*|"") ! :;; (*/*) set "${1%%/*}" "${1#*/}" ;; (*) set "$1" esac&& eval "printf '%s\n' $1${2+/\"\$2\"}"
Improve this answer

7 Comments

This is probably safe -- I haven't found a case it fails for. That said, if we're going to speak to using eval "correctly", I'd argue that Orwellophile's answer follows the better practice: I trust the shell's printf %q to escape things safely more than I trust hand-written validation code to have no bugs.
@Charles Duffy - that's silly. the shell might not have a %q - and printf is a $PATH'd command.
Isn't this question tagged bash? If so, printf is a builtin, and %q is guaranteed to be present.
@Charles Duffy - what version?
@Charles Duffy - that's... pretty early. but i still think its weird that you'd trust a %q arg more than you would code right before your eyes, ive used bash enough before to know not to trust it. try: x=$(printf \\1); [ -n "$x" ] || echo but its not null!
|
0

for some reason when the string is already quoted only perl saves the day

 #val="${val/#\~/$HOME}" # for some reason does not work !! val=$(echo $val|perl -ne 's|~|'$HOME'|g;print')
Improve this answer

Comments

0

I think that

thepath=( ~/abc/def/ghi )

is easier than all the other solutions... or I am missing something? It works even if the path does not really exists.

Improve this answer

1 Comment

Although you're right that this does work for unquoted tildes (and only in bash, which is what was requested), it's my impression that OP is asking about quoted tilde characters; running thepath=( "~/abc/def/ghi" ) doesn't seem to work.
-1

You might find this easier to do in python.

(1) From the unix command line:

python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' ~/fred

Results in:

/Users/someone/fred

(2) Within a bash script as a one-off - save this as test.sh:

#!/usr/bin/env bash thepath=$(python -c 'import os; import sys; print os.path.expanduser(sys.argv[1])' $1) echo $thepath

Running bash ./test.sh results in:

/Users/someone/fred

(3) As a utility - save this as expanduser somewhere on your path, with execute permissions:

#!/usr/bin/env python import sys import os print os.path.expanduser(sys.argv[1])

This could then be used on the command line:

expanduser ~/fred

Or in a script:

#!/usr/bin/env bash thepath=$(expanduser $1) echo $thepath
Improve this answer

3 Comments

Or how about passing only '~' to Python, returning "/home/fred"?
Needs moar quotes. echo $thepath is buggy; needs to be echo "$thepath" to fix the less-uncommon cases (names with tabs or runs of spaces being converted to single spaces; names with globs having them expanded), or printf '%s\n' "$thepath" to fix the uncommon ones too (ie. a file named -n, or a file with backslash literals on an XSI-compliant system). Similarly, thepath=$(expanduser "$1")
...to understand what I meant about backslash literals, see pubs.opengroup.org/onlinepubs/009604599/utilities/echo.html -- POSIX allows echo to behave in a completely implementation-defined manner if any argument contains backslashes; the optional XSI extensions to POSIX mandate default (no -e or -E needed) expansion behaviors for such names.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.