Using system() in parallel in C (Linux)

Question

My code has a large for loop that contains system(). I.e.

for(i=0;i<totalCycles;i++) { <some C code> //Bulk of the cpu time is taken up with the command below system(command_string); <some more C code> }

This for loop can be easily parallelised insofar as an iteration does not depend on a previous iteration. Normally I would use MPI or openMP to parallelise the loop. However, I am not sure how to handle system(). There was a similar question here, but that was in the context of only parallelising system() only, as opposed to system() within a larger loop.

Basile Starynkevitch · Accepted Answer · 2014-09-18 16:38:12Z

Read the system(3) man page, it mentions fork(2) and execl(3), so read these man pages too.

You probably want to fork all (or probably just many enough) your processes in parallel.

You should read Advanced Linux Programming. It has several chapters on the subject (that I won't summarize here).

You'll need to define if you want to get the output of your parallel processes. I guess that you'll need to use pipe(2) and probably poll(2) to have some event loop handling the incoming (or outgoing) data.

BTW, unless your many processes are mostly idle, you probably don't want to fork a lot of processes. Having much more running processes than you have cores in your processors would probably make your system slow down or thrash.

Collectives™ on Stack Overflow

Using system() in parallel in C (Linux)

1 Answer 1

Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Linked

Related