Skip to main content
Tweeted twitter.com/StackUnix/status/1070966287235452928
added 1 character in body
Source Link
Jeff Schaller
  • 68.8k
  • 35
  • 122
  • 266

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo ' > Line 1 > Line B > Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat' 1:Line B 2:Line b 1:Line iii 3:Line III 

So far I have this:

i=0 SOURCES=() TARGETS=() for c in "$@"; do SOURCES+=(">($c > tmp-$i)") TARGETS+=("tmp-$i") i=$((i+1)) done eval tee ${SOURCES[@]} >/dev/null <&0 comm ${TARGETS[@]} 

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non deterministic-deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file wontwon't work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this.? I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo ' > Line 1 > Line B > Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat' 1:Line B 2:Line b 1:Line iii 3:Line III 

So far I have this:

i=0 SOURCES=() TARGETS=() for c in "$@"; do SOURCES+=(">($c > tmp-$i)") TARGETS+=("tmp-$i") i=$((i+1)) done eval tee ${SOURCES[@]} >/dev/null <&0 comm ${TARGETS[@]} 

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file wont work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this. I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo ' > Line 1 > Line B > Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat' 1:Line B 2:Line b 1:Line iii 3:Line III 

So far I have this:

i=0 SOURCES=() TARGETS=() for c in "$@"; do SOURCES+=(">($c > tmp-$i)") TARGETS+=("tmp-$i") i=$((i+1)) done eval tee ${SOURCES[@]} >/dev/null <&0 comm ${TARGETS[@]} 

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non-deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file won't work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this? I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.

Source Link
LambdaBeta
  • 183
  • 2
  • 13

Pass input to multiple commands and compare their outputs

I am trying to pass standard input into multiple commands and compare their outputs. My current attempt seems close, but doesn't quite work - plus it relies on temporary files which I feel would not be necessary.

An example of what I would want my script to do:

$ echo ' > Line 1 > Line B > Line iii' | ./myscript.sh 'sed s/B/b/g' 'sed s/iii/III/' 'cat' 1:Line B 2:Line b 1:Line iii 3:Line III 

So far I have this:

i=0 SOURCES=() TARGETS=() for c in "$@"; do SOURCES+=(">($c > tmp-$i)") TARGETS+=("tmp-$i") i=$((i+1)) done eval tee ${SOURCES[@]} >/dev/null <&0 comm ${TARGETS[@]} 

The issues are:

  • There seems to be a race condition. By the end of execution comm tmp-0 tmp-1 has the desired output (more-or-less) but when executed from the script the output seems non deterministic.
  • This is limited to just 2 inputs, but I need at least 3 (ideally any number)
  • This creates temporary files that I would have to keep track of and delete afterwards, an ideal solution would only use redirection

The constraints are:

  • The input may not be ending. In particular the input could be something like /dev/zero or /dev/urandom, so merely copying the input to a file wont work.
  • The commands may have spaces in them and be fairly complicated themselves
  • I want a line-by-line, in-order comparison.

Any idea how I could go about implementing this. I basically want something like echo $input | tee >(A >?) >(B >?) >(C >?) ?(compare-all-files) if only such a syntax existed.