On Unix-like operating systems, the wget command downloads files served with HTTP, HTTPS, or FTP over a network.

Description

wget is a free utility for non-interactive download of files from the web. It supports HTTP, HTTPS, and FTP protocols, and retrieval through HTTP proxies.

  • Description
  • Overview
  • Installing wget
  • Syntax
  • Basic startup options
  • Logging and input file options
  • Download options
  • Directory options
  • HTTP options
  • HTTPS (SSL/TLS) options
  • FTP options
  • Recursive retrieval options
  • Recursive accept/reject options
  • Files
  • Examples
  • Related commands
  • Linux commands help

wget is non-interactive, meaning that it can work in the background, while the user is not logged on, which allows you to start a retrieval and disconnect from the system, letting wget finish the work. By contrast, most web browsers require constant user interaction, which make transferring a lot of data difficult.

wget can follow links in HTML and XHTML pages and create local versions of remote websites, fully recreating the directory structure of the original site, which is sometimes called “recursive downloading.’’ While doing that, wget respects the Robot Exclusion Standard (robots.txt). wget can be instructed to convert the links in downloaded HTML files to the local files for offline viewing.

wget is designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it keeps retrying until the whole file is retrieved. If the server supports regetting, it instructs the server to continue the download from where it left off.

Overview

The simplest way to use wget is to provide it with the location of a file to download over HTTP. For example, to download the file http://website.com/files/file.zip, this command:

wget http://website.com/files/file.zip

…would download the file into the working directory.

There are many options that allow you to use wget in different ways, for different purposes. These are outlined below.

Installing wget

If your operating system is Ubuntu, or another Debian-based Linux distribution which uses APT for package management, you can install wget with apt-get:

sudo apt-get install wget

For other operating systems, see your package manager’s documentation for information about how to locate the wget binary package and install it. Or, you can install it from source from the GNU official website.

Syntax

wget [option]… [URL]…

Basic startup options

Logging and input file options

Download options

Directory options

HTTP options

HTTPS (SSL/TLS) options

To support encrypted HTTP (HTTPS) downloads, wget must be compiled with an external SSL library, currently OpenSSL. If wget is compiled without SSL support, none of these options are available.

wget -c ftp://sunsite.doc.ic.ac.uk/ls-lR.Z

wget –spider –force-html -i bookmarks.html

wget –no-cookies –header “Cookie: =”

wget –header=‘Accept-Charset: iso-8859-2’ –header=‘Accept-Language: hr’ http://fly.srk.fer.hr/

wget –header=“Host: foo.bar” http://localhost/

wget –save-cookies cookies.txt –post-data ‘user=foo&password=bar’ http://server.com/auth.php

wget –load-cookies cookies.txt -p http://server.com/interesting/article.php

FTP options

Recursive retrieval options

Recursive accept/reject options

Files

Examples

wget https://www.computerhope.com/

Download the default homepage file (index.htm) from www.computerhope.com. The file is saved to the working directory.

wget ftp://gnjilux.srk.fer.hr/*.msg

wget -r -nd –delete-after http://whatever.com/~popular/page/

wget -r -l 2 http:///1.html

wget -r -l 2 -p http:///1.html

wget -r -l 1 -p http:///1.html

wget -r -l 0 -p http:///1.html

wget -p http:///1.html

wget -E -H -k -K -p http:///

wget –ignore-tags=a,area -H -k -K -r http:///

wget –limit-rate=200k http://www.example.org/files/archive.zip

Download the file archive.zip from www.example.org, and limit bandwidth usage of the download to 200k/s.

wget -c http://www.example.org/files/archive.zip

Download archive.zip from example.org, and if a partial download exists in the current directory, resume the download where it left off.

wget -b http://www.example.org/files/archive.zip

Download archive.zip in the background, returning you to the command prompt in the interim.

wget –spider http://www.example.org/files/archive.zip

Uses “web spider” mode to check if a remote file exists. Output resembles the following:

Spider mode enabled. Check if remote file exists. HTTP request sent, awaiting response… 200 OK Length: 1206 (1.2K) [application/zip] Remote file exists.

wget –mirror -p –convert-links -P ./example-mirror http://www.example.org

Download a complete mirror of the website www.example.org to the folder ./example-mirror for local viewing.

wget -Q5m http://www.example.org/files/archive.zip

Stop downloading archive.zip once five megabytes are successfully transferred. This transfer can then later be resumed using the -c option.

curl — Transfer data to or from a server.