wget

What it does

wget is a non-interactive network downloader that retrieves files over HTTP, HTTPS, and FTP. "Non-interactive" is the key word: it's designed to run unattended in scripts, cron jobs, and background processes, with built-in support for retries, resuming partial downloads, and recursively following links to mirror entire sites or directory trees.

Beginner examples

• wget https://example.com/file.zip — download a file, saving it with its original name in the current directory
• wget -O myfile.zip https://example.com/file.zip — download and save under a custom filename
• wget -c https://example.com/bigfile.iso — resume a download that was interrupted
• wget -q https://example.com/file.zip — download quietly, without progress output

wget https://releases.example.com/app-1.2.0.tar.gz

Advanced examples

• Recursively mirror a site or directory: wget -r -np -k https://example.com/docs/
• Save downloads into a specific directory: wget -P ~/Downloads https://example.com/file.zip
• Send a download to the background so it survives closing the terminal: wget -b https://example.com/large.iso
• Throttle bandwidth so a big download doesn't saturate the connection: wget --limit-rate=300k https://example.com/large.iso
• Download a list of URLs from a text file: wget -i urls.txt

wget -r -np -nH --cut-dirs=2 -P ./mirror https://example.com/data/reports/

Common mistakes

• Forgetting -np (no-parent) when recursively downloading a subdirectory — without it, wget -r can climb up and pull in unrelated parts of the site.
• Re-running a plain wget URL after an interrupted download instead of wget -c URL — without -c it starts over from scratch (or creates a file.1 duplicate) instead of resuming.
• Assuming -b shows live progress in the terminal — backgrounded downloads redirect their output to wget-log, so you need to tail -f wget-log to watch progress.
• Not realizing some servers don't support byte-range requests, which means -c can't actually resume and will re-download from the start regardless.

Tips

• Use wget --spider URL to check whether a URL exists and is reachable without actually downloading it — handy in monitoring scripts.
• Combine -c with -b for large, unreliable downloads: it backgrounds the job and will resume cleanly if you rerun the same command later.
• wget -i urls.txt reads one URL per line from a file, useful for batch-downloading many files in one command.

Best practices

• Always use -np (and often -nH --cut-dirs) when mirroring a subdirectory so you don't accidentally pull the entire remote site.
• Rate-limit large or repeated automated downloads with --limit-rate to avoid overwhelming shared networks or triggering abuse detection on the remote server.
• Prefer HTTPS URLs and let wget verify certificates by default; only pass --no-check-certificate as a deliberate, documented exception, never as a habit.

Real-world use cases

• Downloading release artifacts or installers in a CI/CD pipeline or Dockerfile build step.
• Mirroring documentation or datasets from a public directory listing for offline use.
• Resuming a multi-gigabyte ISO or dataset download over an unreliable connection without starting from zero each time it drops.

Common interview questions

• When would you choose wget over curl? When you need straightforward file downloading with built-in recursive mirroring and resume support and don't need curl's broader protocol/API-scripting features; wget's defaults are more download-oriented out of the box.
• How does wget resume a partial download? With -c, it checks the existing local file's size and issues an HTTP Range request asking the server for only the remaining bytes, provided the server honors range requests.
• How do you keep a long-running wget job alive after logging out of an SSH session? Use wget -b URL to background it immediately, or run it under nohup ... & or inside a tmux/screen session.