122

I would like to introduce multithreading feature in my shell script.

I have a script which calls the function read_cfg() with different arguments. Each of these function calls are independent.

Would it be possible to instantiate these function calls (not scripts) parallelly. Please let me how can we achieve that.. ?

Improve this question
7
  • 2
    This is not multithreading -- it's multiprocessing. Each instance is run in a distinct process, copied from the original with fork(). These processes -- unlike threads -- have their own file descriptor tables, and their memory is copy-on-write (so when they change a variable's value, the parent process doesn't see it). Commented Sep 21, 2016 at 22:48
  • bash does not support multithreading. Running functions in background with & will work but each function call will have its own environment, so they cannot comunicate through variables for example. For more details read about linux fork function Commented Dec 30, 2024 at 6:01
  • @CharlesDuffy I don't understand; I'm seeing different comments among this question and its answers saying the same—it's not multithreading, it's multiprocessing—does that mean the processes are all running on the same thread??? What exactly makes this not multithreading and only multiprocessing? Commented Aug 16 at 18:12
  • @crow, if you're using multithreading, then you have multiple threads sharing the same process. If you only have one thread per process, even if there are multiple processes, it's just multiprocessing, not multithreading. And that's the thing -- bash doesn't support threading at all; every use of & runs a fork(), creating a whole new process, and not setting up any threads in either parent or child. Commented Aug 16 at 22:50
  • @crow, ..."processes are all running on the same thread" is backwards. Processes are split into threads; threads are not split into processes. Commented Aug 16 at 22:51

3 Answers 3

Reset to default
246

Sure, just add & after the command:

read_cfg cfgA & read_cfg cfgB & read_cfg cfgC & wait

all those jobs will then run in the background simultaneously. The optional wait command will then wait for all the jobs to finish.

Each command will run in a separate process, so it's technically not "multithreading", but I believe it solves your problem.

Improve this answer
Sign up to request clarification or add additional context in comments.

8 Comments

You should read up the difference between process and thread. What you propose is not multithreading - it involves separate processes for every command.
@TomTom: I certainly know the difference between processes and threads. If you see through the OP's choice of words, I believe he is simply asking whether it's possible to run the commands in parallel (which is possible). I added a note about this to clarify.
It can be slow if you have an high amount of process
@TomTom Are the separate processes not being run on different threads? I suppose they could all be running on one thread that is constantly switching... Is that what happens and thus why you commented?
Threads can not pass processes - by definition. Theads live in ONE process, only. And the process is where i.e. the memory is - all the stuff. So 10 threads in 10 processes is 10 copies of the whole whatever is needed.
@crow there's no such thing as multiple processes "running on one thread" unless you get into things like cooperative multitasking, and in that context you're talking about a completely different kind of "process". UNIX processes are split into threads, not the other way around.
|
40

You can run several copies of your script in parallel, each copy for different input data, e.g. to process all *.cfg files on 4 cores:

 ls *.cfg | xargs -P 4 -n 1 read_cfg.sh

The read_cfg.sh script takes just one parameters (as enforced by -n)

Improve this answer

2 Comments

just a note that you should specify the full path to read_cfg.sh or xargs will say it can't find the file.
Better to use printf '%s\0' *.cfg | xargs -0 ... -- that way this works with filenames with spaces, unprintable characters, etc. See also Why you shouldn't parse the output of ls(1).
30

Bash job control involves multiple processes, not multiple threads.

You can execute a command in background with the & suffix.

You can wait for completion of a background command with the wait command.

You can execute multiple commands in parallel by separating them with |. This provides also a synchronization mechanism, since stdout of a command at left of | is connected to stdin of command at right.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.