Skip to main content
added an alternative snippet with explicit space in it
Source Link

You can use parallel. It's available in most Linux distros.

-jN means that N parallel programs will run. You can tune it to something like -j8 or even more.

cat url-list | parallel -j8 wget {} 

or alternatively

cat url-list | parallel -j8 wget ' ' {} 

cat is reading the file with URLs to download. My file looked liked:

https://XXX/all/Packages/a/abrt-2.10.9-20.el8.x86_64.rpm https://XXX/all/Packages/a/abrt-addon-vmcore-2.10.9-20.el8.x86_64.rpm ... 

The output is then piped to parallel that is responsible for spinning multiple wgets (-jN decide how many). The {} is piped input line.

You can use parallel. It's available in most Linux distros.

-jN means that N parallel programs will run. You can tune it to something like -j8 or even more.

cat url-list | parallel -j8 wget {} 

cat is reading the file with URLs to download. My file looked liked:

https://XXX/all/Packages/a/abrt-2.10.9-20.el8.x86_64.rpm https://XXX/all/Packages/a/abrt-addon-vmcore-2.10.9-20.el8.x86_64.rpm ... 

The output is then piped to parallel that is responsible for spinning multiple wgets (-jN decide how many). The {} is piped input line.

You can use parallel. It's available in most Linux distros.

-jN means that N parallel programs will run. You can tune it to something like -j8 or even more.

cat url-list | parallel -j8 wget {} 

or alternatively

cat url-list | parallel -j8 wget ' ' {} 

cat is reading the file with URLs to download. My file looked liked:

https://XXX/all/Packages/a/abrt-2.10.9-20.el8.x86_64.rpm https://XXX/all/Packages/a/abrt-addon-vmcore-2.10.9-20.el8.x86_64.rpm ... 

The output is then piped to parallel that is responsible for spinning multiple wgets (-jN decide how many). The {} is piped input line.

added 363 characters in body
Source Link

You can use parallel. It's available in most Linux distros.

-jN means that N parallel programs will run. You can tune it to something like -j8 or even more.

cat url-list | parallel -j8 wget {} 

cat is reading the file with URLs to download. My file looked liked:

https://XXX/all/Packages/a/abrt-2.10.9-20.el8.x86_64.rpm https://XXX/all/Packages/a/abrt-addon-vmcore-2.10.9-20.el8.x86_64.rpm ... 

The output is then piped to parallel that is responsible for spinning multiple wgets (-jN decide how many). The {} is piped input line.

You can use parallel. It's available in most Linux distros.

-jN means that N parallel programs will run. You can tune it to something like -j8 or even more.

cat url-list | parallel -j8 wget {} 

You can use parallel. It's available in most Linux distros.

-jN means that N parallel programs will run. You can tune it to something like -j8 or even more.

cat url-list | parallel -j8 wget {} 

cat is reading the file with URLs to download. My file looked liked:

https://XXX/all/Packages/a/abrt-2.10.9-20.el8.x86_64.rpm https://XXX/all/Packages/a/abrt-addon-vmcore-2.10.9-20.el8.x86_64.rpm ... 

The output is then piped to parallel that is responsible for spinning multiple wgets (-jN decide how many). The {} is piped input line.

Source Link

You can use parallel. It's available in most Linux distros.

-jN means that N parallel programs will run. You can tune it to something like -j8 or even more.

cat url-list | parallel -j8 wget {}