Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Universal configuration monitoring and system of record for IT.
Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

May 19, 2015 - A Look At The New Commandlinefu
I've put together a short writeup on what kind of newness you can expect from the next iteration of clfu. Check it out here.
March 2, 2015 - New Management
I'm Jon, I'll be maintaining and improving clfu. Thanks to David for building such a great resource!
Hide

Top Tags

Hide

Functions

Psst. Open beta.

Wow, didn't really expect you to read this far down. The latest iteration of the site is in open beta. It's a gentle open beta-- not in prime-time just yet. It's being hosted over at UpGuard (link) and you are more than welcome to give it a shot. Couple things:

  • » The open beta is running a copy of the database that will not carry over to the final version. Don't post anything you don't mind losing.
  • » If you wish to use your user account, you will probably need to reset your password.
Your feedback is appreciated via the form on the beta page. Thanks! -Jon & CLFU Team

Commands tagged wget from sorted by
Terminal - Commands tagged wget - 89 results
wget --no-check-certificate https://code.google.com/p/msysgit/downloads/list -O - 2>nul | sed -n "0,/.*\(\/\/msysgit.googlecode.com\/files\/Git-.*\.exe\).*/s//http:\1/p" | wget -i - -O Git-Latest.exe
2012-11-14 08:17:50
User: michfield
Functions: sed wget
Tags: git windows wget
-1

This command should be copy-pasted in Windows, but very similar one will work on Linux.

It uses wget and sed.

function ip-where { wget -qO- -U Mozilla http://www.ip-adress.com/ip_tracer/$1 | html2text -nobs -style pretty | sed -n /^$1/,/^$/p;}
2012-10-22 21:39:53
User: tox2ik
Functions: sed wget
0

Tries to avoid the fragile nature of scrapers by looking for user-input in the output as opposed to markup or headers on the web site.

telnet v4address.com
while pgrep wget || sudo shutdown -P now; do sleep 1m; done
cat urls.txt | wget -i- -T 10 -t 3 --waitretry 1
curl -Is http://osswin.sourceforge.net/ | grep Mod
2012-05-01 21:42:42
User: Gunni
Functions: grep
1

I is for headers only

s is for silence

curl -Is outputs ONLY headers the pipe and grep is to filter them to Modified only..

wget -S --spider http://osswin.sourceforge.net/ 2>&1 | grep Mod
2012-04-18 03:43:33
User: dmmst19
Functions: grep wget
6

I used to use the Firefox "View page info" feature a lot to determine how stale the web page I was looking at was. Now that I use mostly Chrome I miss that feature, so here is a command line alternative using wget. The -S says to display the server response, the --spider says to not download any files/pages, just fetch the header. The output goes to stderr, so to grep it you use 2>&1 to combine the stderr stream with stdout, the pipe that to grep for Last-Modified.

You can use curl instead if you have it installed, like this:

curl --head -s http://osswin.sourceforge.net | grep Mod
for i in $(seq 1 `curl http://megatokyo.com 2>/dev/null|grep current|cut -f6 -d\"`);do wget http://megatokyo.com/`curl http://megatokyo.com/strip/${i} 2>/dev/null|grep src=\"strips\/|cut -f4 -d\"`;done
2012-03-04 22:52:36
User: akira88
Functions: cut grep seq wget
Tags: wget comic
1

A simple script for download all the MegaTokyo strips from the first to the last one

echo -n $(curl -Ss http://icanhazip.com) | xclip
2012-02-17 16:58:40
User: red_five
Functions: echo
0

Gets the IP and sticks it into the middle-mouse-click buffer

wget http://icanhazip.com -qO-
wget -nd -r -l 2 -A jpg,jpeg,png,gif http://website-url.com
wget -A mp3,mpg,mpeg,avi -r -l 3 http://www.site.com/
expandurl() { wget -S $1 2>&1 | grep ^Location; }
2011-10-18 18:50:54
User: atoponce
Functions: grep wget
Tags: wget
0

This shell function uses wget(1) to show what site a shortened URL is pointing to, even if there are many nested shortened URLs. This is a great way to test whether or not the shortened URL is sending you to a malicious site, or somewhere nasty that you don't want to visit. The sample output is from:

expandurl http://t.co/LDWqmtDM
isgd() { /usr/bin/wget -qO - "http://is.gd/create.php?format=simple&url=$1" ;}
wgetall () { wget -r -l2 -nd -Nc -A.$@ $@ }
2011-09-28 09:43:25
Functions: wget
0

Recursively download all files of a certain type down to two levels, ignoring directory structure and local duplicates.

Usage:

wgetall mp3 http://example.com/download/

wget -O - -q http://checkip.dyndns.org/ | cut -d':' -f2 | cut -d'<' -f1| cut -c2-
2011-09-17 13:42:01
User: ztank1013
Functions: cut wget
-2

This is just a "cut" addicted variant of the previous unixmonkey24730 command...

wget http://checkip.dyndns.org/ -q -O - | grep -Eo '\<[[:digit:]]{1,3}(\.[[:digit:]]{1,3}){3}\>'
curl -sm1 http://www.website.com/ | grep -o 'http://[^"]*jpg' | sort -u | wget -qT1 -i-
wget -q -O - http://www.perl.org/get.html | grep -m1 '\.tar\.gz' | sed 's/.*perl-//; s/\.tar\.gz.*//'
for i in `seq -w 1 50`; do wget --continue \ http://commandline.org.uk/images/posts/animal/$i.jpg; done
echo ".mode tabs select host, case when host glob '.*' then 'TRUE' else 'FALSE' end, path, case when isSecure then 'TRUE' else 'FALSE' end, expiry, name, value from moz_cookies;" | sqlite3 ~/.mozilla/firefox/*.default/cookies.sqlite
2011-08-15 14:49:47
User: euridice
Functions: echo
6

useful to use after with the --load-cookies option of wget

wget --spider $URL 2>&1 | awk '/Length/ {print $2}'
2011-07-03 00:14:58
User: d3Xt3r
Functions: awk wget
5

- Where $URL is the URL of the file.

- Replace the $2 by $3 at the end to get a human-readable size.

Credits to svanberg @ ArchLinux forums for original idea.

Edit: Replaced command with better version by FRUiT. (removed unnecessary grep)

wget -r -A .pdf -l 5 -nH --no-parent http://example.com
2011-06-09 17:17:03
User: houghi
Functions: wget
Tags: wget pdf
7

See man wget if you want linked files and not only those hosted on the website.

curl -s http://example.com | grep -o -P "<a.*href.*>" | grep -o "http.*.pdf" | xargs -d"\n" -n1 wget -c
2011-06-09 14:42:46
User: b_t
Functions: grep wget xargs
0

This example command fetches 'example.com' webpage and then fetches+saves all PDF files listed (linked to) on that webpage.

[*Note: of course there are no PDFs on example.com. This is just an example]

wget --spider -o wget.log -e robots=off --wait 1 -r -p http://www.example.com
2011-04-05 13:42:14
User: lele
Functions: wget
-1

This will visit recursively all linked urls starting from the specified URL. It won't save anything locally and it will produce a detailed log.

Useful to find broken links in your site. It ignores robots.txt, so just use it on a site you own!