2

I'm trying to use socat on two systems in order to send a multivolume tar from one system to the other. Multivolume is an absolute requirement as I'm trying to shift potentially petabytes of data to a mounted tape archive.

Ultimately, I'm want to do something like this:

# on system1: socat PIPE:/tmp/pipe1 SYSTEM:"ssh system2 socat - PIPE\:/tmp/pipe2" & tar -M -L 10G -F /tmp/chvol -cvf /tmp/pipe1 <source-path> & # on system2: cat /tmp/pipe2 > file001.tar cat /tmp/pipe2 > file002.tar ...

(The chvol script just echos the path /tmp/pipe1.)

As I've not been able to get this working yet, I'm running the following much-simplified test on one single system:

socat PIPE:/tmp/pipe1 PIPE:/tmp/pipe2 & sh -c 'while sleep 1; do echo "sending date ..."; date > /tmp/pipe1; done' & sh -c 'while sleep 1; do echo "slurping ..."; cat < /tmp/pipe2; done' &

which is meant to simulate a writer process repeatedly opening, writing, closing and a reader process repeatedly opening, reading until EOF, closing.

But this doesn't behave as I expected: the writer is seemingly happy, but the reader reads all the accumulated text in one go and thereafter remains blocked in a cat < ... call.

Can anybody explain (1) what is actually happening in the much-simplified test and (2) what I should be doing instead?

I have tried various shut-* options on both pipes (on the assumption that the problem is in the propogation of EOF) and done a lot of googling.

Improve this question

1 Answer 1

Reset to default
2

Your problem is that you have not specified you want unidirectional pipes. The socat man page explains how in this case PIPE: is like an echo.

What probably happened is that when you first wrote the date into the fifo 1, your socat read it, wrote it to fifo 2, then noticed there was input on fifo 2 so read it and wrote it to fifo 1, and so on busy looping until the first cat on fifo 2 managed to empty it before socat read it. Thereafter this first cat will continue reading the fifo, never seeing an end-of-file, and you can continue writing the date into the fifo.

You can see this if you add the -v option to socat to show the i/o.

Adding -u for unidirectional i/o makes it what you probably wanted, but now the socat will exit after the date is written and read, so you will need a loop for it:

while socat -v -u PIPE:/tmp/pipe1 PIPE:/tmp/pipe2; do echo new; done &

With your tar version you could put the socat command inside the /tmp/chvol script instead.

If you have a real tape drive, you might also look at a solution using nbd-server to export a block device over the network.

Improve this answer
1
  • I didn't realise that socat's reading of data from fifo1 and writing it to fifo2 made fifo2 readable by socat; I assumed that my own reader process (cat, in my much-simplified example) would be the only process reading. But your explanation fits! Many thanks Meuh! Commented Apr 21, 2017 at 11:11

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.