106

I have a bash script which calls several long-running processes. I want to capture the output of those calls into variables for processing reasons. However, because these are long running processes, I would like the output of the rsync calls to be displayed in the console in real-time and not after the fact.

To this end, I have found a way of doing it but it relies on outputting the text to /dev/stderr. I feel that outputting to /dev/stderr is not a good way of doing things.

VAR1=$(for i in {1..5}; do sleep 1; echo $i; done | tee /dev/stderr) VAR2=$(rsync -r -t --out-format='%n%L' --delete -s /path/source1/ /path/target1 | tee /dev/stderr) VAR3=$(rsync -r -t --out-format='%n%L' --delete -s /path/source2/ /path/target2 | tee /dev/stderr)

In the example above, I am calling rsync a few times and I want to see the file names as they are processed, but in the end I still want the output in a variable because I will be parsing it later.

Is there a 'cleaner' way of accomplishing this?

If it makes a difference, I am using Ubuntu 12.04, bash 4.2.24.

Improve this question
0

5 Answers 5

Reset to default
98

Duplicate &1 in your shell (in my example to 5) and use &5 in the subshell (so that you will write to stdout (&1) of the parent shell):

exec 5>&1 FF=$(echo aaa|tee >(cat - >&5)) echo $FF

This will print "aaa" two times, once because of the echo in the subshell, and the second time it prints the value of the variable.

In your code:

exec 5>&1 VAR1=$(for i in {1..5}; do sleep 1; echo $i; done | tee >(cat - >&5)) # use the value of VAR1
Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

Shouldn't the descriptor be closed in the parent shell?
@akhan: I assume that's exec 5>&- then?
How about FF=$(echo aaa | tee /dev/tty)? Source
This does not preserve output colors.
@user1011471, when you write tee >(inner_cmd...), the shell opens a pipe to the inner command, such that the pipe's write side is also inherited by the outer command (tee), typically under the descriptor 63. The outer command has no idea the shell did anything funny, it gets a file name argument /dev/fd/63. en.wikipedia.org/wiki/Process_substitution Don't know why you got this error, but it's unrelated to inner command writing to descriptor 5.
|
51

Op De Cirkel's answer has the right idea. It can be simplified even more (avoiding use of cat):

exec 5>&1 FF=$(echo aaa|tee /dev/fd/5) echo $FF
Improve this answer

8 Comments

Wouldn't /dev/fd/5 be OS specific?
I had been wondering if you could just use tee /dev/fd/1, but that doesn't work because the output still gets captured by $(). So in case anyone else is wondering the same thing, it is necessary to use an extra file descriptor (like 5).
We could go simplifying even further and making this a oneliner without the exec: { FF=$(echo aaa|tee /dev/fd/5); } 5>&1 The braces allow for the redirection to happen before the subshell command is run, while $FF still remains in the scope of the current shell (that wouldn't work with normal brackets ( ). This way there's even no need to close FD 5 afterwards, which is a overlooked hygienic habit.
@akhan No it wouldn't, bash is said to be emulating this path should it not exist in the OS by itself.
If this is run using sudo -u <other non-root user> <script> then Op De Cirkel's answer works but this answer does not. Writing to /dev/fd/5 is equivalent to writing directly to the terminal. /dev/fd/5 is a symlink to the /dev/pts/ file for the terminal, which is owned by the user that originally logged in and is not writable by the sudo'd user. However, cat - >&5 writes to a file descriptor that is opened by bash within the process (which is not the same as writing to /dev/fd/5). This file descriptor forwards the write through each parent process, avoiding any permissions issues.
|
23

Here's an example capturing stderr, stdout, and the command's exit code. This is building on the answer by Russell Davis.

exec 5>&1 FF=$(ls /taco/ 2>&1 |tee /dev/fd/5; exit ${PIPESTATUS[0]}) exit_code=$? echo "$FF" echo "Exit Code: $exit_code"

If the folder /taco/ exists, this will capture its contents. If the folder doesn't exist, it will capture an error message and the exit code will be 2.

If you omit 2>&1then only stdout will be captured.

Improve this answer

Comments

12

If by "the console" you mean your current TTY, try

variable=$(command with options | tee /dev/tty)

This is slightly dubious practice because people who try to use this sometimes are surprised when the output lands somewhere unexpected when they don't have a TTY (cron jobs etc).

Improve this answer

3 Comments

It does not work in Docker containers. tee: /dev/tty: No such device or address
I don't think that's generally true. It works for me here in a Debian Docker container just fine.
You can use tty=$(readlink /proc/$$/fd/2) in docker and then NEW=$(echo "blah" | tee "$tty")
7

You can use more than three file descriptors. Try here:

http://tldp.org/LDP/abs/html/io-redirection.html

"Each open file gets assigned a file descriptor. [2] The file descriptors for stdin, stdout, and stderr are 0, 1, and 2, respectively. For opening additional files, there remain descriptors 3 to 9. It is sometimes useful to assign one of these additional file descriptors to stdin, stdout, or stderr as a temporary duplicate link."

The point is whether it's worth to make script more complicated just to achieve this result. Actually it's not really wrong, the way you do it.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.