15 Practical examples of wget command

wget is a Linux/UNIX command line file downloader. The wget is a free tool for non-interactive download of files from the internet. It supports HTTP, HTTPS, and FTP protocols to connect server and download files, in addition to retrieval through HTTP proxies. Linux tool wget is non-interactive, meaning it may function in the background, which enables the feature to use without login on the system for the user.

Within this informative article, we’ll discuss various cases of wget command to use.

Suggested Reading: Linux wget command help and Examples

Single file download

The wget utility simple downloads a single file and store it into the current working directory. This command also shows file size, downloading time, and download progress, as shown below.

# wget https://linuxconcept.com/download/pdf/wget-guide.pdf
--2019-05-20 11:28:30--  https://linuxconcept.com/download/pdf/wget-guide.pdf
Resolving linuxconcept.com... 203.112.245.11, 2001:3240:212:2::d
Connecting to linuxconcept.com|203.112.245.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-pdf]
Saving to: wget-guide.pdf
100%[===================================================================================>] 446,966     60.0K/s   in 7.4s
2019-05-20 11:28:38 (58.9 KB/s) - wget-guide.pdf

Multiple file download

The wget tool allows us to download multiple files at once, here you can see how to download multiple files using wget command.

# wget https://linuxconcept.com/download/pdf/wget-guide.pdf ftp://linuxconcept.com/download/pdf/wget-help.pdf
--2019-05-02 12:11:16--  https://linuxconcept.com/download/pdf/wget-guide.pdf
Resolving linuxconcept.com... 203.112.245.11, 2001:3240:212:2::d
Connecting to linuxconcept.com|203.112.245.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-pdf]
Saving to: wget-guide.pdf
100%[===================================================================================>] 446,966     56.7K/s   in 7.6s
2019-05-02 12:11:29 (57.1 KB/s) - wget-guide.pdf
--2019-05-02 12:11:29--  ftp://linuxconcept.com/download/pdf/wget-help.pdf
           => wget-help.pdf
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /gnu/wget ... done.
==> SIZE wget-help.pdf ... 65
==> PASV ... done.    ==> RETR wget-help.pdf ... done.
Length: 65 (unauthoritative)
100%[===================================================================================>] 65          --.-K/s   in 0s
2019-05-02 12:11:33 (2.66 MB/s) - wget-help.pdf
FINISHED --2019-05-02 12:11:33--
Downloaded: 2 files, 437K in 7.6s (57.1 KB/s)

Save downloaded file with different name

To save the downloaded file with different name use “-O” option. Here in below example, I save the downloaded file “wget-guide-1.0.3.4.pdf” with the name “wget.pdf”.

# wgetO wget.pdf https://linuxconcept.com/download/pdf/wget-guide-1.0.3.4.pdf
--2019-05-20 11:28:30--  https://linuxconcept.com/download/pdf/wget-guide-1.0.3.4.pdf
Resolving linuxconcept.com... 203.112.245.11, 2001:3240:212:2::d
Connecting to linuxconcept.com|203.112.245.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-pdf]
Saving to: wget.pdf
100%[===================================================================================>] 446,966     60.0K/s   in 7.4s
2019-05-20 11:28:38 (58.9 KB/s) - wget.pdf

Read download links from a file

Suppose you have to download multiple files more than 10 or 20. In this case, we have the option to write all downloading links into a text file and use single wget command to download all files, where all downloading links will automatically read from the file and downloaded into the current directory.

# wget -i /wget/tmp.txt
--2019-05-20 12:34:12--  https://linuxconcept.com/download/wget/wget-guide.pdf ftp://linuxconcept.com/download/wget/wget.zip
Resolving ftp.gnu.org... 203.112.245.11, 2001:3240:212:2::d
Connecting to ftp.gnu.org|203.112.245.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 446966 (436K) [application/x-pdf]
Saving to: wget.pdf
100%[===================================================================================>] 446,966     35.0K/s   in 10s
2019-05-20 12:34:23 (42.7 KB/s) – wget.pdf
--2019-05-20 12:34:23--  https://linuxconcept.com/download/centos/CentOS-6.3-x86_64-LiveDVD.iso
Resolving mirrors.hns.net.in... 203.112.245.11, 2001:3240:212:2::d
Connecting to mirrors.hns.net.in|203.112.245.11|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1761607680 (1.6G) [application/octet-stream]
Saving to: CentOS-6.3-x86_64-LiveDVD.iso
 45%[========================================== 

Resume partial Download File

Sometimes downloading get failed due to the big size of a file, or network disconnection. To avoid these use “-c” command line option with wget command.

By using the “-c” option you can resume download, so I recommended you whenever downloading big size file use “-c” to support resume for downloading files as shown below.

# wget -c http://mirrors.hns.net.in/centos/7/isos/x86_64/CentOS-7.0-x86_64-LiveDVD.iso
--2019-05-02 12:46:57--  http://mirrors.hns.net.in/centos/7/isos/x86_64/CentOS-7.0-x86_64-LiveDVD.iso
Resolving mirrors.hns.net.in... 111.91.91.34, 2401:4800::111:91:91:34
Connecting to mirrors.hns.net.in|111.91.91.34|:80... connected.
HTTP request sent, awaiting response... 206 Partial Content
Length: 1761607680 (1.6G), 1758132697 (1.6G) remaining [application/octet-stream]
Saving to: CentOS-6.3-x86_64-LiveDVD.iso
 57% [=================================================                                  ] 4,077,262   99.0K/s  eta 4h 27m ^

Download files in the background

You can use the wget tool with the “-b” option to send the process into the background immediately after starting the download.

# wget -b ftp://linuxconcept.com/download/wget-guide.pdf
Continuing in background, pid 2457.

Download files with given download speed

You have the option to use wget tool to download file with your desired speed; the only thing is your client and server network should support that speed.

To set download speed use “-limit-rate” option, as shown below.

# wget -c --limit-rate=500k  ftp://linuxconcept.com/download/debian/debian-8.0.5-i386-DVD-1.iso
--2019-05-22 13:16:21--  ftp://linuxconcept.com/download/debian/debian-8.0.5-i386-DVD-1.iso
           => debian-8.0.5-i386-DVD-1.iso
esolving ftp.iinet.net.au... 203.112.245.11
Connecting to ftp.iinet.net.au|203.112.245.11|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done.    ==> PWD ... done.
==> TYPE I ... done.  ==> CWD (1) /download/debian/ ... done.
==> SIZE debian-6.0.5-i386-DVD-1.iso ... 4691312640
==> PASV ... done.    ==> REST 2825236 ... done.
==> RETR debian-8.0.5-i386-DVD-1.iso ... done.
Length: 4688487404 (4.4G), 4685662168 (4.4G) remaining (unauthoritative)
 0% [

Use wget with download retry attempts

You have the option to increase or decrease retry attempt for downloading files. You can set a retry attempt using “-tries” option, as shown below:

# wget --tries=50 https://linuxconcept.com/download/wget/wget-guide.pdf

Redirect command execution logs into log file

When you execute a command in the background, it is useful if you are writing logs in a file for future reference.

The wget tool has capabilities to redirect logs in a file using the “-o” option, as shown below:

# wget -o wget-download.log https://linuxconcept.com/download/wget/wget-guide.pdf

The above command writes all logs into the file “wget-download.log” in the current directory.

Download full website to mirroring site for offline view

You can download a full website with wget command, as shown below:

# wget --mirror -p --convert-links -P ./<save-directory> website-url

Where,

–mirror is used to mirroring a site
-p help to download all file to use website offline as online.
–convert-link is use to convert all links into a document for local viewing.
-P ./<save-directory> is to save all files into the “save-directory” path.

Exclude file types while downloading

When you use the wget tool to download files, you also have the option to exclude any file type from the downloading list using “—reject” option as shown below:

# wget --reject=png linuxconcept.com/download

The above command will download all files located into “download” directory except any “png” files.

Set download quota with wget command

You can also set download quota with wget tool using the “-Q” option, as shown below:

# wgetQ20m -i download.txt

The quota will never effect on single file download. It is only applicable when it goes into the recursive mode. So if you use above command, it will automatically abort download when the quota is exceeding.

Use username and password to download protected site or files

There is an option to pass username and password with wget command to download password protected files or site. It work on http and ftp both protocols with option “- -http-user”, “- -http-password”, “- -ftp-user”, and “- -ftp-password” relatively, as shown below:

# wget --http-user=linuxconcept --http-password=password https://linuxconcept.com/download/centos/CentOS-7.0-x86_64-LiveDVD.iso

# wget --ftp-user=linuxconcept --ftp-password=password ftp://ftp.linuxconcept.com/debian/debian-8.0.5-i386-DVD-1.iso

How to find wget version

To know, which version of wget tool installed, use “- -version” option.

# wget --version

Get help for wget command on terminal

To know more about wget utility, use “- -help” option on the terminal.

# wget --help

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Related Articles