0

Apologies if this is a basic question - I'm stuck trying to solve a larger problem, and it's come down to how a shell script is invoked - directly (shellScript.sh) or using sh shellScript.sh.

Here's a model for the problem:

When I execute on bash:

cat <(echo 'Hello')

I see the output

Hello

But when I use:

sh -c "cat <(echo 'Hello')"

I see errors:

sh: -c: line 0: syntax error near unexpected token `(' sh: -c: line 0: `cat <(echo 'Hello')'

I've tried escaping the <, ( and ) in various combinations, but I don't see the output anywhere. What am I missing here?

My actual problem is that I'm passing a <() as an input argument to a python script within a shell script, and while it works fine when I invoke the shell script using just the name, if I use sh to invoke it, I get errors similar to what I've shown above.

Thank you!

Improve this question
3
  • 1
    try bash -c "cat <(echo 'Hello')". If you're using Debian/Ubuntu, you most likely have dash as your /bin/sh. see: lwn.net/Articles/343924 Commented May 9, 2017 at 20:27
  • It doesn't even matter if sh is linked to bash, when invoked as sh it operates in "POSIX mode" and wouldn't support that feature Commented May 9, 2017 at 20:44
  • @steeldriver, bash disables <(...) when invoked as sh. Commented May 9, 2017 at 21:08

1 Answer 1

Reset to default
6

Process substitution is a feature that originated in the Korn shell in the 80s (first documented in ksh86). At the time, it was only available on systems that had support for /dev/fd/<n> files.

Later, the feature was added to zsh (from the start: 1990) and bash (in 1993). zsh was using temporary named pipes to implement it, while bash was using /dev/fd/<n> where available and named pipes otherwise. zsh switched to using /dev/fd/<n> where available in 2.6-beta17 in 1996.

Support for process substitution via named pipes on systems without /dev/fd was only added to ksh in ksh93u+ in 2012. The public domain clone of ksh doesn't support it.

To my knowledge, no other Bourne-like shell supports it (rc, es, fish, non-Bourne-like shells support it but with a different syntax). yash has a <(...) construct, but that's for process redirection.

While quite useful, the feature was never standardized by POSIX. So, one can't expect to find it in sh, so shouldn't use it in a sh script.

Though the behaviour for <(...) is unspecified in POSIX, (so there would be no harm in retaining it), bash disables the feature when called as sh or when called with POSIXLY_CORRECT=1 in its environment.

So, if you have a script that uses <(...), you should use a shell that supports the feature to interpret it like zsh, bash or AT&T ksh (of course, you need to make sure the rest of the syntax of the script is also compatible with that shell).

In any case:

cat <(cmd)

Can be written:

cmd | cat

Or just

cmd

For a command other than cat (that needs to be passed data via a file given as argument), on systems with /dev/fd/x, you can always do:

something | that-cmd /dev/stdin

Or if you need that-cmd's stdin to be preserved:

{ something 3<&- | that-cmd /dev/fd/4 4<&0 <&3 3<&-; } 3<&0

Where 3<&0 duplicates fd 0 onto fd 3 for the whole {...} command group so as to make the corresponding resource it's opened on available to that-cmd.

In the pipeline in that command group, we close that fd 3 with 3<&- for the left hand side (something) as it doesn't need it, and for the right hand side (that-cmd):

  • with 4<&0 we copy fd 0 (the reading end of the pipe) to fd 4, so that-cmd can open the pipe on /dev/fd/4
  • then move (copy + close) fd 3 (our original stdin) to fd 0, so that-cmd's stdin is the original stdin (the one of the command group).
Improve this answer
8
  • Thank you for the detailed explanation. I have a few follow up questions/remarks. My cat/echo example is just that - a base example. My actual use case involves a python script that takes in 3 inputs, one of which is being passed at the moment as a <(...). This py script is being used within a shell script, and if I run the shell script using just its name (in which case it uses the bash interpreter), it works fine. Commented May 10, 2017 at 0:27
  • Now, when I use sh script.sh, it uses bash too, because that's the default shell on the HPC that I use. Is there any way I can ensure the script is invoked only using bash and not anything else? As in, can I check within the script if the script was invoked using bash script.sh or sh script.sh? Commented May 10, 2017 at 0:29
  • @Ram, see What is the portable (POSIX) way to achieve process substitution? for instance. You'd want to stop giving your script a sh extension if it's not a sh script, and use a shebang with the correct interpreter like #! /path/to/bash -. If you want to check that the shell is bash in your script, try [ -n "$BASH_VERSION" ], and whether it's in POSIX mode with [ -o posix ]. Commented May 10, 2017 at 6:50
  • I found another of your answers very useful to my situation: unix.stackexchange.com/a/71137/135331 I'm going to see if I can incorporate that into my script. Commented May 10, 2017 at 12:39
  • 1
    @TheQuark, see if the edit makes it clearer. Commands in pipelines are run concurrently, redirections are applied for each part independently, something 3<&- only closes fd 3 for something. Commented Jul 30 at 6:03

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.