How do I use exec 3>myfifo in a script, and not have echo foo>&3 close the pipe?

Question

Why can't I use exec 3>myfifo in the same manner in a bash script as I can in my terminal?

I'm using named pipes to turn an awk filter into a simple "server", that should be able to take text input from clients, filter it, and flush on NUL.

In terminal 1, the server is running like this:

$ mkfifo to_server from_server; $ while true; do # Really, this awk script BEGIN's with reading in a huge file, # thus the client-server model awk '{sub("wrong", "correct");print;} /\0/ {fflush();}' <to_server >from_server; echo "restarting..."; done

And I've got a simple script that should put input, ending with a NUL, into the input-pipe, and read from the output pipe, and then exit, without sending an EOF to the server (we don't want the server to restart):

#!/bin/bash # According to http://mywiki.wooledge.org/BashFAQ/085 , using # `exec 3>mypipe; echo foo >&3;` instead of `echo foo >mypipe` # should ensure that the pipe does not get the EOF which closes it: exec 3>to_server; exec 4<from_server; cat >&3; echo -e '\0' >&3; while read -rd '' <&4; do echo -n "$REPLY"; break; done;

Now, if I in terminal 2 do $ echo This is wrong | bash client.sh, I get This is correct back, but terminal 1 shows me that the server restarts! If I run the commands from client.sh from within terminal 2, however, it does not restart.

It seems to be related to the exec commands, since I can also do

$ exec 3>to_server; exec 4<from_server; $ echo "This is wrong" | sh client.sh

and it does not restart. If I then

$ exec 3>&-; exec 4<&-

(which of course restarts it once) and do

$ echo "This is wrong" | sh client.sh

it restarts every time. So it seems the exec commands in the script have no effect. However, putting ls /proc/$$/fd/ after the exec commands in the script shows that they do in fact point to the correct pipes.

What am I missing here?

Have you tried ls /proc/$$/fd/ before the exec? My guess is that handles 3&4 are already in use, and whoever owns them closes your pipe from under you. Also, I'd try replacing handles with something like 103 & 104 in your original code and see what happens. — atzz
– atzz, Commented Mar 21, 2011 at 11:14

unhammer · Accepted Answer · 2011-03-21 11:10:50Z

I think I figured it out!

The exec commands do work, but bash itself closes all open file descriptors on exiting from the script. Adding sleep 5 to the end of the client shows that it takes 5 seconds for the server to finally shut down.

So the solution is just to open some file descriptor to my named pipes from some other terminal, and just keep them open, e.g. in terminal 3:

$ exec 3>to_server; exec 4<from_server $ # keep open for as long as server is open

or, in the server terminal/script itself:

while true; do # Really, this awk script BEGIN's with reading in a huge file, # thus the client-server model awk '{sub("wrong", "correct");print;} /\0/ {fflush();}' <to_server >from_server & AWKPID=$! exec 3>to_server; exec 4<from_server wait "$AWKPID" echo "restarting..."; done

nuro · Accepted Answer · 2011-04-01 19:32:36Z

Here's a somewhat different version that processes (client-side) input in five-line blocks.

# using gawk for Mac OS X from: http://rudix.org/packages-ghi.html#gawk # server rm -v to_server from_server mkfifo to_server from_server ( while true; do (exec gawk '{sub("wrong", "correct");print;} /\0/ {fflush();}' <to_server >from_server) & bgpid=$! exec 3>to_server; exec 4<from_server wait "$bgpid" echo "restarting..." done ) & # client.sh #!/bin/bash exec 3>to_server exec 4<from_server n=0 clientserver() { n=0 ( while read -rd '' <&4; do echo -n "$REPLY"; break; done; IFS="" read -r -d $'\n' <&4 lines && printf 'found a trailing newline in from_server fifo \n' "$lines" ) & bgpid=$! printf '%s\n' "${lines[@]}" >&3 printf '%b' '\000\n' >&3 wait $bgpid unset -v lines return 0 } while IFS="" read -r -d $'\n' line; do n=$((n+=1)) lines[$((n-1))]="$line" if [[ $n -eq 5 ]]; then clientserver fi done if [[ ${#lines[@]} -gt 0 ]]; then clientserver fi exit 0 # test serverpid=$! echo This is wrong | bash client.sh printf '%s\n' {1..1007} | bash client.sh kill -TERM $serverpid

I see, the problem is that if you send in a huge file with no flushes, it'll hang (because of filling up the pipe buffer) before it ever gets to the reading part which empties the pipe.

Collectives™ on Stack Overflow

How do I use exec 3>myfifo in a script, and not have echo foo>&3 close the pipe?

2 Answers 2

Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Linked

Related