5

This code snippet is from Advanced Bash Scripting Guide.

# Redirecting only stderr to a pipe. exec 3>&1 # Save current "value" of stdout. ls -l 2>&1 >&3 3>&- | grep bad 3>&- # Close fd 3 for 'grep' (but not 'ls'). # ^^^^ ^^^^ exec 3>&- # Now close it for the remainder of the script. # Thanks, S.C. 

The comments explains the code closes fd 3 only for 'grep'. but the closes fd 3 twice. Why we need to do like this? Is it wrong to close fd 3 only one time for 'grep' like this?

ls -l 2>&1 >&3 | grep bad 3>&- 

3 Answers 3

5

Neither ls nor grep need anything open on their fd 3 (they don't use that fd), so it's better practice to close it (release the resources you don't need). We only use that fd 3 to be able to restore ls stdout to the original one (before executing ls).

Remember that in a pipe line, all commands run concurrently in their own process. Redirections apply for each separately, so closing the fd 3 for one cannot close it for the other one.

It won't make much difference in practice in this case to close or not to close other than you might reach a limit on the number of file descriptors sooner if you don't.

In other cases (commands themselves starting other processes for instance), that may hold resources and cause problems. Think for instance if stdout is a pipe for instance, a background process may end up inheriting that fd, preventing any reader from that pipe to see EOF until that process has returned.

A better way to write it would be:

{ ls -l 2>&1 >&3 3>&- | grep bad 3>&-; } 3>&1 

That way, the fd 3 is only redirected temporarily for the duration of that command group (and restored afterwards or closed).

See the difference:

$ { ls -l /proc/self/fd 2>&1 >&3 3>&- | grep bad 3>&-; } 3>&1 total 0 lrwx------ 1 stephane stephane 64 Apr 2 09:29 0 -> /dev/pts/4 lrwx------ 1 stephane stephane 64 Apr 2 09:29 1 -> /dev/pts/4 l-wx------ 1 stephane stephane 64 Apr 2 09:29 2 -> pipe:[575886] lr-x------ 1 stephane stephane 64 Apr 2 09:29 3 -> /proc/20918/fd/ $ { ls -l /proc/self/fd 2>&1 >&3 | grep bad 3>&-; } 3>&1 total 0 lrwx------ 1 stephane stephane 64 Apr 2 09:29 0 -> /dev/pts/4 lrwx------ 1 stephane stephane 64 Apr 2 09:29 1 -> /dev/pts/4 l-wx------ 1 stephane stephane 64 Apr 2 09:29 2 -> pipe:[575900] lrwx------ 1 stephane stephane 64 Apr 2 09:29 3 -> /dev/pts/4 lr-x------ 1 stephane stephane 64 Apr 2 09:29 4 -> /proc/20926/fd/ 

In the second invocation, ls had its fd 3 also open to the terminal (which was open on stdout at the time I ran the pipeline) for no good reason.

Note that in ksh93, with exec, you don't need to close those fds as fds other than 0, 1, 2 are automatically closed upon executing a command.

$ ksh93 -c 'exec 3>&1; ls -l /dev/fd/' total 0 lrwx------ 1 stephane stephane 64 Apr 2 09:34 0 -> /dev/pts/16 lrwx------ 1 stephane stephane 64 Apr 2 09:34 1 -> /dev/pts/16 lrwx------ 1 stephane stephane 64 Apr 2 09:34 2 -> /dev/pts/16 lr-x------ 1 stephane stephane 64 Apr 2 09:34 3 -> /proc/21105/fd $ ksh93 -c 'exec 3>&1; ls -l /dev/fd/ 3>&3' total 0 lrwx------ 1 stephane stephane 64 Apr 2 09:34 0 -> /dev/pts/16 lrwx------ 1 stephane stephane 64 Apr 2 09:34 1 -> /dev/pts/16 lrwx------ 1 stephane stephane 64 Apr 2 09:34 2 -> /dev/pts/16 lrwx------ 1 stephane stephane 64 Apr 2 09:34 3 -> /dev/pts/16 lr-x------ 1 stephane stephane 64 Apr 2 09:34 4 -> /proc/21108/fd 

I don't know why it says (but not 'ls') above, sounds like a mistake (possibly mine ;-).

3
  • 3
    Hey so what does #Thanks, S.C mean there? Commented Apr 2, 2014 at 7:44
  • Do I understand correctly, that pipe is created before fd3 is closed by 3>&- (for ls part). Is it described somewhere in which order descriptors and/or pipes are created? Commented May 22, 2014 at 19:33
  • 2
    @vyegorov, the pipe has to be created before the fork (otherwise, the two commands couldn't both access it), so before the redictions. The order is pipe, fork, and then redirection applied in each part of the pipe (in the different process), and then the commands are executed (after the forks, the difference processes work independently, so one might still be doing redirections while the other one has alread executed its command). Commented May 22, 2014 at 19:45
3

Close fd3 twice because you fork two subshell in script, each subshell is inherited and copy parent's file descriptor. Close fd3 in each subshell doesn't affect others (and the parent, too).

So the comment line is very unclear and causes misleading.

If you want to redirect only stder to pipe, you can use process substitution:

ls -l 2> >(grep bad) 

or swap stderr and stdout:

ls -l 3>&1 1>&2 2>&3 | grep bad 
0

For anyone else arriving here because of the question summary, as opposed understanding the code snippet from "Advanced Bash Scripting Guide", I found the other answers hard to reason through. The following approach worked for my purposes and I find it easier to read and understand its intention:

$ ls -l 2>&1 1>&- | grep bad 

To me this says that I want to ignore stdout by closing it, then I want to redirect stderr to stdout so I can pipe it onto the next command.

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.