File Transfer - Wget
About
wget is a headless browser command line.
It's most used to download files from the Web but you can also download/mirror a whole website.
It supports the HTTP, HTTPS, and FTP protocols
Articles Related
Windows
Windows 10 PowerShell has also curl available.
Installation
- From Gnu Windows, download the binary program (example: wget.exe)
- Optionally put it into the Git_Home\bin directory at
Git_Home\usr\bin
Management
Download to a specific file name
- -O specify the target file name
wget <file.ext> -O /path/to/folder/file.ext
Download to a specific folder
- -P define the download folder
wget <file.ext> -P /path/to/folder
Download a file with basic authentication
What is and how works the Basic Authentication scheme? (HTTP)
wget --http-user=login --http-password=password https://example.com/file.pdf
Download a whole web site
wget
--recursive \ # Download the whole site.
--page-requisites \ # Get all assets/elements (CSS/JS/images).
--adjust-extension \ # Save files with .html on the end.
--span-hosts \ # Include necessary assets from offsite as well.
--convert-links \ # Update links to still work in the static version.
--restrict-file-names=windows \ # Modify filenames to work in Windows as well.
--domains yoursite.com \ # Do not follow links outside this domain.
--no-parent \ # Don't follow links outside the directory you pass in.
https://yoursite.com/whatever/path # The start URL to download
Options:
-r - recursive
-m, --mirror Turns on recursion and time-stamping, sets infinite recursion depth, and keeps FTP directory listings.
-p, --page-requisites Get all images, etc. needed to display HTML page.
-E, --adjust-extension Save HTML/CSS files with .html/.css extensions.
-k, --convert-links Make links in downloaded HTML point to local files.
It can be used to pre-render