Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Universal configuration monitoring and system of record for IT.
Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

May 19, 2015 - A Look At The New Commandlinefu
I've put together a short writeup on what kind of newness you can expect from the next iteration of clfu. Check it out here.
March 2, 2015 - New Management
I'm Jon, I'll be maintaining and improving clfu. Thanks to David for building such a great resource!
Hide

Top Tags

Hide

Functions

Hide

Credits

Commands using wget from sorted by
Terminal - Commands using wget - 252 results
wget -q -O - "$@" <url>
wget --server-response --spider http://www.example.com/
2009-03-31 18:49:14
User: penpen
Functions: wget
5

Let me suggest using wget for obtaining the HTTP header only as the last resort because it generates considerable textual overhead. The first ellipsis of the sample output stands for

Spider mode enabled. Check if remote file exists.

--2009-03-31 20:42:46-- http://www.example.com/

Resolving www.example.com... 208.77.188.166

Connecting to www.example.com|208.77.188.166|:80... connected.

HTTP request sent, awaiting response...

and the second one looks for

Length: 438 [text/html]

Remote file exists and could contain further links,

but recursion is disabled -- not retrieving.

wget --http-user=YourUsername --http-password=YourPassword http://YourWebsiteUrl:2082/getbackup/backup-YourWebsiteUrl-`date +"%-m-%d-%Y"`.tar.gz
2009-03-31 17:50:41
User: nadavkav
Functions: wget
4

this will connect to your hosted website service through the cPanel interface and use its backup tool to backup and download the entire website, locally.

(do not forget to replace : YourUsername , YourPassword and YourWebsiteUrl for it to work )

wget -c -t 1 --load-cookies ~/.cookies/rapidshare <URL>
2009-03-28 09:13:35
User: cammarin
Functions: wget
6

The download content part.

NOTE: the '-c' seems to not work very well and the download stuck at 99% sometimes. Just finish wget with no problem. Also, the download may restart after complete. You can also cancel. I don't know if it is a wget or Rapidshare glitch since I don't have problems with Megaupload, for example.

UPDATE: as pointed by roebek the restart glitch can be solved by the "-t 1" option. Thanks a lot.

wget --save-cookies ~/.cookies/rapidshare --post-data "login=USERNAME&password=PASSWORD" -O - https://ssl.rapidshare.com/cgi-bin/premiumzone.cgi > /dev/null
2009-03-28 09:12:02
User: cammarin
Functions: wget
4

In order to do that, first you need to save a cookie file with your account info. These commands do it (maybe you need to create the '.cookies' dir before). Also, you need to check the "Direct downloads" option on the Premium Zone >> Settings tab.

You need to do this once (as long you maintain the file or your Rapidshare Premium account).

wget -qO- whatismyip.org
wget -q http://xyz.gpg -O- | sudo apt-key add -
2009-03-25 12:18:36
User: gnuyoga
Functions: sudo wget
Tags: wget apt-key
4

when we add a new package to a aptitude (the debian package manager) we need to add the gpg, otherwise it will show warning / error for missing key

wget -q http://xyz.gpg -O- | sudo apt-key add -
2009-03-25 12:18:29
User: gnuyoga
Functions: sudo wget
Tags: apt-key
2

when we add a new package to a aptitude (the debian package manager) we need to add the gpg, otherwise it will show warning / error for missing key

wget -r ftp://user:pass@ftp.example.com
2009-03-09 19:39:30
User: movaxes
Functions: wget
Tags: wget
6

If the username includes an @ you can use this one:

wget -r --user=username_here --password=pass_here ftp://ftp.example.com

wget -S -O/dev/null "INSERT_URL_HERE" 2>&1 | grep Server
2009-03-09 06:54:54
User: asmoore82
Functions: grep wget
8

the good:

Server: Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.4 with Suhosin-Patch

the bad:

Server: Microsoft-IIS/6.0

and the ugly:

Server: Apache/2.2.10 (Win32) mod_ssl/2.2.10 OpenSSL/0.9.8i PHP/5.2.6

wget -c -v -S -T 100 --tries=0 `curl -s http://ms1.espectador.com/ podcast/espectador/la_venganza_sera_terrible.xml | grep -v xml | grep link | sed 's/]*>//g'`
2009-03-04 13:12:28
User: fmdlc
Functions: grep link sed wget
-3

This download a complete audio podcast

vlc --one-instance --playlist-enqueue -q $(while read netcast; do wget -q $netcast -O - |grep enclosure | tr '\r' '\n' | tr \' \" | sed -n 's/.*url="\([^"]*\)".*/\1/p'|head -n1; done <netcast.txt)
2009-03-03 04:26:01
User: tomwsmf
Functions: read sed tr wget
1

This is a quick line to stream in the latest offerings of your favorite netcasts/podcasts. You will need to have a file named netcast.txt in the directory you run this from. This file should have one and only one of your netcast's/podcst's url per line.

When run the line grabs the offering on the top of the netcast/podcast stack and end it over , quietly, to vlc.

Since I move around computers during the day I wanted an easy way to listen to my daily dose of news and such without having to worry about downloading to whatever machine I am on. This is just a quick grab and stream of whats current.

Future plans... have the list of netcasts be read from the web. possibly an rss or such. I use greader so there might be a way to use it as the source so as not to have to muck with multiple lists

alias wordpress='mkdir wordpress && cd wordpress && wget http://wordpress.org/latest.tar.gz && tar -xvzf latest.tar.gz && mv wordpress/* . && rm -rf latest.tar.gz wordpress && cp wp-config-sample.php wp-config.php'
wget -r -l1 -H -t1 -nd -N -np -A.mp3 -erobots=off -i ~/sourceurls.txt
2009-02-19 03:52:47
User: tomhiggins
Functions: wget
1

This was gotten from http://www.veen.com/jeff/archives/000573.html. The line will grab all the mp3 files on the urls listed in text file sourceurls.txt (one url per line) . A much more complete breakdown of the line can be found at the web site mentioned above.

wget -qO - http://infiltrated.net/blacklisted|awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}'
2009-02-18 16:08:23
User: sil
Functions: wget
33

Blacklisted is a compiled list of all known dirty hosts (botnets, spammers, bruteforcers, etc.) which is updated on an hourly basis. This command will get the list and create the rules for you, if you want them automatically blocked, append |sh to the end of the command line. It's a more practical solution to block all and allow in specifics however, there are many who don't or can't do this which is where this script will come in handy. For those using ipfw, a quick fix would be {print "add deny ip from "$1" to any}. Posted in the sample output are the top two entries. Be advised the blacklisted file itself filters out RFC1918 addresses (10.x.x.x, 172.16-31.x.x, 192.168.x.x) however, it is advisable you check/parse the list before you implement the rules

wget -qO - snubster.com|sed -n '65p'|awk 'gsub(/<span><br>.*/,"")&&1'|perl -p -e 's:myScroller1.addItem\("<span class=atHeaderOrange>::g;s:</span> <span class=snubFontSmall>::g;s:&quot;:":g;s:^:\n:g;s:$:\n:'
2009-02-18 15:05:13
User: sil
Functions: wget
-2

I've got this posted in one of my .bash_profiles for humor whenever I log in.

wget --random-wait -r -p -e robots=off -U mozilla http://www.example.com
2009-02-17 22:53:07
User: starchox
Functions: wget
157

-p parameter tells wget to include all files, including images.

-e robots=off you don't want wget to obey by the robots.txt file

-U mozilla as your browsers identity.

--random-wait to let wget chose a random number of seconds to wait, avoid get into black list.

Other Useful wget Parameters:

--limit-rate=20k limits the rate at which it downloads files.

-b continues wget after logging out.

-o $HOME/wget_log.txt logs the output

wget --recursive --page-requisites --convert-links www.moyagraphix.co.za
2009-02-16 10:57:44
User: RonaldConco
Functions: wget
16

Very nice command when you want to download a site locally to your machine, including images, css and javascript

wget http://search.twitter.com/trends.json -O - --quiet | ruby -rubygems -e 'require "json";require "yaml"; puts YAML.dump(JSON.parse($stdin.gets))'
wget -p --convert-links http://www.foo.com
2009-02-13 19:02:08
User: damniel
Functions: wget
8

The example will create a directory called "

Caveats: @imports of css files will not be converted.

wget -qO - "http://www.tarball.com/tarball.gz" | tar zxvf -
wget -qO - http://myip.dk/ | egrep -m1 -o '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'
seq -f"ftp://ftp.vim.org/pub/vim/patches/7.1/7.1.%03g" 176 240 | xargs -I {} wget -c {};
2009-02-06 03:19:06
User: liupeng
Functions: seq wget xargs
3

Seq allows you to define printf like formating by specified with -f, %03g is actually tells seq I got three digits, fill the blank digits with 0, and the range is from 176 to 240.

echo -n "search> ";read QUERY && wget -O - `wget -O - -U "Mozilla/5.0" "http://images.google.com/images?q=${QUERY}" 2>/dev/null |sed -e 's/","http/\n","http/g' |awk -F \" '{print $3}' |grep -i http: |head -1` > "$QUERY"
2009-02-05 19:50:53
User: wwest4
Functions: echo wget
2

prompts for a search term and then pulls down the first result from google images

wget -O - http://www.commandlinefu.com/commands/browse/rss 2>/dev/null | awk '/\s*<title/ {z=match($0, /CDATA\[([^\]]*)\]/, b);print b[1]} /\s*<description/ {c=match($0, /code>(.*)<\/code>/, d);print d[1]"\n"} '