Howto do floating point compasiosn in an if-statement within a GNU parallel block?

Question

I want to run a batch process in parallel. For this I pipe a list to parallel. When I've an if-statement, that compares two floating point numbers (taken form here), the code doesn't run anymore. How can this be solved.

LIMIT=25 ps | parallel -j2 ' echo "Do stuff for {} to determine NUM" NUM=33.3333 # set to demonstrate if (( $(echo "$NUM > $LIMIT" | bc -l) )); then echo "react..." fi echo "Do stuff..." '

Prints:

Do stuff for \ \ PID\ TTY\ \ \ \ \ \ \ \ \ \ TIME\ CMD to determine NUM Do stuff... (standard_in) 2: syntax error #... snipp

Add a shebang and then paste your script there: shellcheck.net — Cyrus
– Cyrus, Commented Jun 2, 2018 at 16:55
Depending on what you are doing and the overhead of running external programs like bc, it may be faster to just do everything serially in a language with proper floating-point support. — chepner
– chepner, Commented Jun 2, 2018 at 18:54
In my real code, I call an external (serial) program that runs for 2-3 minutes (instead of echo "react..."). Is there no way to get this done? Do I need to put the parallel stuff in an extra shell script? — dani
– dani, Commented Jun 2, 2018 at 20:59

rici · Accepted Answer · 2018-06-03 01:28:57Z

LIMIT is not set inside parallel shell. Running echo "$NUM > $LIMIT" | bc -l exapands to echo "123 > " | bc -l which results in syntax error reported by bc. You need to export/pass/put it's value to the shell run from inside parallel. Try this:

LIMIT=25 ps | parallel -j2 ' LIMIT="'"$LIMIT"'" echo "Do stuff for {} to determine NUM" NUM=33.3333 # set to demonstrate if (( $(echo "$NUM > $LIMIT" | bc -l) )); then echo "react..." fi echo "Do stuff..." '

Or better use env_parallel, designed for such problems.

Side note: GNU parallel was designed for executing jobs in parallel using one or more computers. For scripts running on one computer it is better to stick with the xargs command, which is more commonly available (so you don't need to install some package each time you move your script to another machine).

Ole Tange · Accepted Answer · 2018-06-03 07:38:20Z

While GNU Parallel is designed to deal correctly with commands spanning multiple lines, I personally find that hard to read. I prefer using a function:

doit() { arg="$1" echo "Do stuff for $a to determine NUM" NUM=33.3333 # set to demonstrate if (( $(echo "$NUM > $LIMIT" | bc -l) )); then echo "react..." fi echo "Do stuff..." } export -f doit LIMIT=25 export LIMIT ps | parallel -j2 doit

Instead of the exports you can use env_parallel:

ps | env_parallel -j2 doit

If your environment is too big, use env_parallel --session before starting:

#!/bin/bash env_parallel --session # Define functions and variables _after_ running --session doit() { [...] } LIMIT=25 ps | env_parallel -j2 doit

Collectives™ on Stack Overflow

Howto do floating point compasiosn in an if-statement within a GNU parallel block?

2 Answers 2

1 Comment

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Linked

Related