3

For parallelizing gzip compression:

parallel gzip ::: myfile_*

does the job but how to pass gzip options such as -r or -9

I tried parallel gzip -r -9 ::: myfile_* and parallel gzip ::: 9 r myfile_*

but it doesn't work.

when I tried parallel "gzip -9 -r" ::: myfile_*

I get this error message :

gzip: compressed data not written to a terminal. Use -f to force compression

Also the -r switch for recursively adding directories is not working.

....

Similarly for other commands: how to pass the options while using parallel ?

Improve this question

3 Answers 3

Reset to default
2

You have the correct syntax:

parallel gzip -r -9 ::: myfile_*

So something else is wrong. What is the output of

parallel --version

You can find more about GNU Parallel at: http://www.gnu.org/s/parallel/

You can install GNU Parallel in just 10 seconds with:

wget -O - pi.dk/3 | sh

Watch the intro video on

http://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

my version is 20121222. I downloaded from fedora repositories. perhaps i should download the latest verstion
Version 20121222 is fine. But Fedora is known to mess up GNU Parallel for their users: stackoverflow.com/questions/16448887/… So please post the full output of 'parallel --version' - not just the version number.
cool... so you are the developer.. well it gives the following output: WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu. GNU parallel 20121222 Copyright (C) 2007,2008,2009,2010,2011,2012 Ole Tange and Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. GNU parallel comes with no warranty. Web site: http://www.gnu.org/software/parallel When using GNU Parallel for a publication please cite: O. Tange (2011): GNU Parallel - The Command-Line Power Too
The warning should give you a hint: WARNING: YOU ARE USING --tollef. IF THINGS ARE ACTING WEIRD USE --gnu.
And did you follow it? (i.e. either remove --tollef or add --gnu)
|
1

(I don't think this question belongs here. Maybe superuser.com?)

parallel gzip -r -9 ::: * worked fine for me, going into directories and all. I am using parallel version 20130622.

Note that with this approach, each directory will be a single task. You may instead want to pipe the output of find to parallel to give each file separately to parallel.

Improve this answer

Comments

1

Have you tried the --gnu flag for parallel??

parallel -j+0 --gnu "command"....

In some systems (like Ubuntu) is disabled by default.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.