4

I mean the use of GNU env or BSD env command in the form of:

env [name=value ...] [utility [args ...]] 

Looks like there is no way to escape special characters in value part but I am not pretty sure how env is implemented to parse the value part.

I know there are many ways to do this by shell's feature but I want to pass literal string without shell's help (a bit like execute env by exec). That is to say, I need to find some kind of literal string format with newline and supported by env command. For example:

env FOO=LITERAL_STRING ruby -e 'puts ENV["FOO"]' 

Here the LITERAL_STRING should contain a literal string with newline and env should understand that format.

With the above command, the expected output should be:

hello world 

I wonder if it is possible. I would appreciate for your help.

Environment

  • env

    I use BSD env so it can't print the version. Don't know if man can help:

    $ man env | tail -n1 BSD April 17, 2008 BSD 
  • OS

    $ sw_vers ProductName: macOS ProductVersion: 11.6 BuildVersion: 20G165 
8
  • 2
    What makes you think that export FOO="hello\nworld" works any differently than env BAR="hello\nworld"? Commented Oct 14, 2021 at 10:46
  • What is your actual Q? What is not working when you use env? Commented Oct 14, 2021 at 10:47
  • 1
    Not on my system. Can you please edit your post and add these commands and their output? Also make sure to tell us your operating system and your shell. Oh, and also tell us what output you are expecting. export FOO="hello\nworld" shouldn't insert a literal newline. Commented Oct 14, 2021 at 10:55
  • 1
    Neither on Debian 11. Commented Oct 14, 2021 at 10:56
  • 1
    But why would you expect env FOO="hello\nworld" to add a newline? That isn't how it works. Both env FOO="hello\nworld" ruby -e 'puts ENV["FOO"]' and export FOO="hello\nworld"; ruby -e 'puts ENV["FOO"]' will print hello\nworld and not an actual newline. That is the expected behavior. Commented Oct 14, 2021 at 11:00

4 Answers 4

11

Your understanding is incorrect. When you store the content "hello\nworld" into a variable, the \n is interpreted literally. Only if you invoke tools such as printf or echo with -e flag, they expand those backslash sequences when printing to console.

In your case, you want to pass the variable to the environment with newline character expanded, suggest using the ANSI C style $'...' quoting operator of ksh93, now supported by several other shells including bash and zsh for this:

env FOO=$'hello\nworld' ruby -e 'puts ENV["FOO"]' 

Or if portability is an issue, a trivial solution is to put those newlines where you want them

f="hello world" env FOO="$f" ruby -e 'puts ENV["FOO"]' 
13
  • You used one of the shell feature called substitution that would process argements before forking the env process. That's kind of cheating in my opinion. Commented Oct 14, 2021 at 11:09
  • Not sure what substitution you are referring to here. I've made an alternate solution too. Also what you mean by cheating in this context? This is how you would do if you wanted an expanded newline character to be present in the variable before passing to your program. Commented Oct 14, 2021 at 11:12
  • 6
    @WeihangJian the main point is that var="foo\nbar" never means "foo, then a newline and then bar". It always means "foo, then a slash, then an n and then bar". That's just how it works. Commented Oct 14, 2021 at 11:14
  • 6
    @WeihangJian, you already determined that env itself doesn't interpret backslash escapes or such, so the only way you can pass a newline to it is to pass it as a bare raw newline. To do that from the shell, the simple options are either the $'...' or using a hard newline. The intermediate variable is of no consequence, you could use env FOO='hello<newline>world' cmd... directly (with a raw newline in place the <newline> placeholder, of course). Please don't accuse people of cheating when they try to provide you with ways to do what you need to do. Commented Oct 14, 2021 at 11:17
  • 1
    @WeihangJian Thanks for those updates. BTW, the second example of my answer should work for you. Commented Oct 15, 2021 at 3:21
5

GNU's and FreeBSD's env have a -S ("split") option[1] which splits its argument on spaces and then interprets a lot of escapes in the resulting strings, including but not limited to \n:

$ env -S 'foo=bar\nquux printenv foo' bar quux $ env -S 'foo=bar\nquux' printenv foo bar quux 

[1] its main use is for #! ... shebang lines, but it could be used in other places where a command line is expected, yet an ad-hoc parser rather than a shell is used to interpret it, like the -e option of most vte-based terminals.

4

The env command, when executed takes a number of arguments --just like any command-- from the execve() system call. The process executing it will do something like:

execve("/path/to/env", ["env", "-options", "var=value", "cmd", "arg"], environ); 

And env, in turn, will do (generally in the same process):

execve("/path/to/cmd", ["cmd", "arg"], modified_environ); 

where modified_environ is environ but with var=value appended to it, or if there was already one or more string starting with var= in environ, the first of them (at least) replaced with var=value.

Those arguments are NUL-terminated arrays of bytes, so the only bytes they can't contain is the NUL byte. Same applies to environment variables whose neither name nor value can contain NUL bytes.

env, upon startup, processes its arguments from first to last.

The ones at the start that start with - would be taken as options.

- alone is also treated as the ancient form of -i to ignore environ.

The ones following options (or -- that can be used to mark the end of options) that contain at least one = will be considered as env var strings (to put in modified_environ above). The first non-option argument that doesn't contain a = character is considered as the command name (which will be passed as first argument to the command and also used to look-up the path of the executable in $PATH).

Any argument after that will be passed as arguments to the command even if they start with - or contain = characters.

So for env itself, only the - and = characters are special; - only during option processing, and = only until the command name is found.

To be able to pass an environment variable name that starts with - or a command name that starts with -, you can just use --:

execve("/usr/bin/env", ["env", "--", "-var-=value", "cmd"], environ); 
execve("/usr/bin/env", ["env", "--", "-cmd-"], environ); 

With several env implementations, - alone following -- would still not be taken as a command name. So if you wanted to call the command called -, you'd need to pass at least one environment variable beforehand:

execve("/usr/bin/env", ["env", "--", "dummy=", "-", "args"], environ); 

(or use /path/to/- or ./- instead of - and bypath $PATH look up of that - command).

env however doesn't let you run a command whose name contains = characters and can't pass a string without = characters in the modified_environ. There is no way to escape the = character in those cases.

But in your case of env FOO=LITERAL_STRING, there is no character in LITERAL_STRING other than the NUL byte that would be a problem as far as env itself is concerned.

Now, when writing code in some language to execute that env command, there will likely be characters in that language that can't be entered literally.

For instance, in perl, you'd do:

exec "env", "--", "-text-=hello\nworld\n", "printenv", "--", "-text-"; 

With the newline character expressed as \n inside double quotes, though you could also do:

exec qw(env --), q(-text-=hello world ), qw(printenv -- -text-); 

To pass strings that contain newline characters.

In the syntax of Bourne-like shell languages, the newline character is not special when inside single or double quotes (like for the q(...) or qq(...) quoting operators of perl), so you could do:

exec env -- "-text-=hello world " printenv -- -text 

It would be a different matter in the C shell, or in the rc shell where "..." is not a quoting operator.

Your env FOO=LITERAL_STRING ruby -e 'puts ENV["FOO"]' looks like code in a shell language. That syntax would be valid with most shells as virtually all understand space as word delimiters, and '...' as a quoting operator.

When you omit the exec at the start, shells run the command in a child process, and then wait for that process to terminate or be suspended.

If that env FOO=LITERAL_STRING ruby -e 'puts ENV["FOO"]' is for a language that doesn't allow entering newline characters literally (likely not a shell) nor using some form of encoding (\n, %0a, &#10;...), but apparently supports '...' as a quoting operator like most shells do, beside the BSD env -S trick already suggested by @UncleBilly, you can invoke an interpreter or a language that can execute commands and allows specifying newline some encoded way, such as perl, ruby, python, ksh93, zsh, bash...

Since you already have ruby:

ruby -e 'exec "env", "FOO=a\n\n\nb", *ARGV' -- ruby -e 'puts ENV["FOO"]' 

But any descent programming language including ruby can set environment variables by themselves, so you don't need env:

ruby -e 'ENV["FOO"]="a\n\n\nb"; exec *ARGV' -- ruby -e 'puts ENV["FOO"]' 
0
2

Use a literal newline in the value:

$ env FOO="hello world" ruby -e 'puts ENV["FOO"]' 

Output:

hello world 

If you have a string with \n you want to convert to an actual newline, use a utility such as printf, just as you would anywhere else:

$ env FOO="$(printf "%b" 'hello\nworld')" ruby -e 'puts ENV["FOO"]' 
hello world 

env isn't special in that respect (in fact, env only ever sees the real newline, because it's expanded before the process is started).

1
  • 1
    +1 beware though that with env FOO="$(printf "%b" 'hello\nworld\n\n\n')", all the trailing newline characters would be removed by the command substitution. bash and zsh support printf -v tmp %b "$string_with_escapes"; env FOO="$tmp" ... but in zsh, you'd rather use env FOO=${(g::)string ... (echo-style, though \c not handled) or ${(g:o:)string} (C/usual-style) Commented Oct 15, 2021 at 5:10

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.