I was trying to download some 150-200 zip files from a http URL by finding each file & clicking on it to download.  I soon realized how boring it is using standard download support in browsers.

This is something I am supposed to do periodically

I know that there are some browser extensions available that could do this pretty easily, but i wondered if i could do this via command line & later on automate it somehow.

This is when i recalled wget.

I downloaded it for my machine from here

After some experimentation i finalized following working set for myself:

wget “http://somewebsiteyoutarget/folderx/foldery/” -r -A zip

Options Explanation:

-r : recursively digs the folder until it exhausts. There are options for specifying recursion levels too (see wget -h)

-A : you can specify extensions you want to download. In my case these are zip files only.

Since these files were quite big in size, it took some time to download these files. Occasionally my internet connection broke & i had to restart things again.

But wget has solution for these problems as well (see wget -h). I explored the help and found -c & -t options pretty usefull for crappy connections.

So now my command looks like this:

wget “http://somewebsiteyoutarget/folderx/foldery/” -r -A zip -c -t

Wget rocks (full marks) ! : o ) !

Tags: wget linux unix filedownloads http