Overview

Wget retrieves content from web servers

Requirements

  • Homebrew installed
  • Wget installed

Install wget on MacOS using Homebrew

brew install wget

Outcome

Offline copy/mirror of a website

Example: Mirror SpaceX.com

  • Open Terminal.app (or iTerm.app)
  • Run Wget to Mirror SpaceX.com locally:
wget --recursive --page-requisites --adjust-extension --span-hosts --convert-links --domains spacex.com --no-parent spacex.com

Wget Explained

# One liner

wget --recursive --page-requisites --adjust-extension --span-hosts --convert-links --restrict-file-names=windows --domains yoursite.com --no-parent yoursite.com

# Explained

wget \

--recursive \ # Download the whole site.

--page-requisites \ # Get all assets/elements (CSS/JS/images).

--adjust-extension \ # Save files with .html on the end.

--span-hosts \ # Include necessary assets from offsite as well.

--convert-links \ # Update links to still work in the static version.

--restrict-file-names=windows \ # Modify filenames to work in Windows as well.

--domains yoursite.com \ # Do not follow links outside this domain.

--no-parent \ # Don't follow links outside the directory you pass in.

yoursite.com/whatever/path # The URL to download

Related pages