Download Files In Parallel

UpdatedAugust 3, 2025

•1 min read

I am a developer from Malaysia. I work with PHP most of the time, recently I fell in love with Go. When I am not working, I will be ballroom dancing :-)

Part of seriesProTips

This is a quick way to download files from different URLs in parallel.

List all URLs in a file `urls.txt`

https://example.com/file1.zip
https://example.com/file2.zip
https://example.com/file3.zip
https://example.com/file4.zip

One-liner download using `wget`

cat urls.txt | xargs -P 4 -n 1 wget

-P <max-procs> sets number of parallel processes. Use 0 for maximum.

One-liner download using `wget`

cat urls.txt | xargs -P 0 -n 1 curl -O

-P <max-procs> sets number of parallel processes. Use 0 for maximum.

#protip #cli #commands #wget #curl #oneliner

12 views

Comments

Join the discussion

No comments yet. Be the first to comment.

ProTips

Part 1 of 9

Tips and tricks to make life easier. Work smart, be a lazy programmer.

Up next

Why You Should Stop Using Global npm install -g (and What to Do Instead)

When developing Node.js applications, you might be tempted to use npm install -g for convenience. However, global npm package installations can lead to significant issues, especially when managing Node.js applications rather than just project depende...

More from this blog

Da Ma Cai 大马彩 API

TL;DR Scarp draw result, host on GitHub, retrieve it like API at damacai.hongineer.com. Data Scraping I have previously shared a technique for scraping Da Ma Cai draw result. Now I finally have time t

Feb 26, 20261 min read17

Monitor MySQL Replication Status with a Bash Script

This is a Bash script that check if the replica (slave) health status. If there is an issue, it will exist with non-zero status. This is useful when used with monitoring tools like [Monit](https://mmo

Feb 24, 20262 min read15

Monitor MySQL Replication Status with a Bash Script

Cheatsheet & Examples: chmod

chmod is a command-line utility used to change the access permissions of file system objects such as files and directories. It allows you to control who can read, write, and execute those objects. Changing Permissions Using Octal Notation Example Usa...

Jan 23, 20263 min read9

Cheatsheet & Examples: chown

chown changes the ownership of files and directories. This involves modifying the user and/or group associated with a file, determining who has access and permissions. Changing File Ownership to a Specific User Example Usage: chown user file.txt What...

Jan 20, 20263 min read6

Cheatsheet & Examples: usermod

The usermod command modifies a user account's properties on a Linux system. It allows administrators to change various attributes associated with a user, such as the user's login name, home directory, shell, groups, and password expiry information. C...

Jan 16, 20263 min read11

Hong's Tech Blog

110 posts

The blog is older than you know. I prefer counting from the emergence of one integral anomaly to the emergence of the next, in which case this is the forth version.

Command Palette

List all URLs in a file urls.txt

One-liner download using wget

One-liner download using wget

Comments

ProTips

Why You Should Stop Using Global npm install -g (and What to Do Instead)

More from this blog

List all URLs in a file `urls.txt`

One-liner download using `wget`

One-liner download using `wget`