Commands using wget sorted by votes

Commands using wget (286)

sorted by

Wget Command to Download Full Recursive Version of Web Page

The example will create a directory called " Caveats: @imports of css files will not be converted.
This is sample output - yours may be different.
8

wget -p --convert-links http://www.foo.com

damniel · 2009-02-13 19:02:08 11
Poke a Webserver to see what it's powered by.

the good: Server: Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.4 with Suhosin-Patch the bad: Server: Microsoft-IIS/6.0 and the ugly: Server: Apache/2.2.10 (Win32) mod_ssl/2.2.10 OpenSSL/0.9.8i PHP/5.2.6
This is sample output - yours may be different.
8

wget -S -O/dev/null "INSERT_URL_HERE" 2>&1 | grep Server

asmoore82 · 2009-03-09 06:54:54 7
Download free e-books

Mask the user agent as firefox, recursively download 2 levels deep from a span host with a maximum of 1 redirection, use random wait time and dump all pdf files to myBooksFolder without creating any other directories. Host will have no way of knowing that this is a grabber script.
This is sample output - yours may be different.
8

wget -erobots=off --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3" -H -r -l2 --max-redirect=1 -w 5 --random-wait -PmyBooksFolder -nd --no-parent -A.pdf http://URL

unixmonkey8504 · 2010-03-01 20:22:44 29
Google Translate

allow multiword translations Show Sample Output
This is sample output - yours may be different.
```
$ translate en de I am looking for a table
```
8

translate() { lng1="$1";lng2="$2";shift;shift; wget -qO- "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=${@// /+}&langpair=$lng1|$lng2" | sed 's/.*"translatedText":"$[^"]*$".*}/\1\n/'; }

bandie91 · 2010-05-02 22:15:30 36
Get all files of particular type (say, PDF) listed on some wegpage (say, example.com)

See man wget if you want linked files and not only those hosted on the website.
This is sample output - yours may be different.
8

wget -r -A .pdf -l 5 -nH --no-parent http://example.com

houghi · 2011-06-09 17:17:03 6
Download an entire ftp directory using wget

If the username includes an @ you can use this one: wget -r --user=username_here --password=pass_here ftp://ftp.example.com
This is sample output - yours may be different.
7

wget -r ftp://user:pass@ftp.example.com

movaxes · 2009-03-09 19:39:30 13
Dump HTTP header using wget

Let me suggest using wget for obtaining the HTTP header only as the last resort because it generates considerable textual overhead. The first ellipsis of the sample output stands for Spider mode enabled. Check if remote file exists. --2009-03-31 20:42:46-- http://www.example.com/ Resolving www.example.com... 208.77.188.166 Connecting to www.example.com|208.77.188.166|:80... connected. HTTP request sent, awaiting response... and the second one looks for Length: 438 [text/html] Remote file exists and could contain further links, but recursion is disabled -- not retrieving. Show Sample Output
This is sample output - yours may be different.
```
&#8230;
  HTTP/1.1 200 OK
  Date: Tue, 31 Mar 2009 18:42:46 GMT
  Server: Apache/2.2.3 (CentOS)
  Last-Modified: Tue, 15 Nov 2005 13:24:10 GMT
  ETag: "b80f4-1b6-80bfd280"
  Accept-Ranges: bytes
  Content-Length: 438
  Connection: close
  Content-Type: text/html; charset=UTF-8
&#8230;
```
7

wget --server-response --spider http://www.example.com/

penpen · 2009-03-31 18:49:14 15
Block the 6700 worst spamhosts

The above url contains over 6700 of the common ad websites. The command just pastes these into your /etc/hosts. Show Sample Output
This is sample output - yours may be different.
```
-
```
7

wget -q -O - http://someonewhocares.org/hosts/ | grep ^127 >> /etc/hosts

torrid · 2009-12-14 17:11:16 8
Limit bandwidth usage by any program

Trickle is a voluntary, cooperative bandwidth shaper. it works entirely in userland and is very easy to use. The most simple application is to limit the bandwidth usage of programs.
This is sample output - yours may be different.
7

trickle -d 60 wget http://very.big/file

point_to_null · 2010-03-29 06:55:30 5
Lets Tux say the random fact. [add it to .bashrc to see it in new terminal window]
This is sample output - yours may be different.
7

wget randomfunfacts.com -O - 2>/dev/null|grep \<strong\>|sed "s;^.*<i>$.*$</i>.*$;\1;"|cowsay -f tux

unixmonkey10612 · 2010-07-01 18:31:36 3
Retrieve a random command from the commandlinefu.com API

Seeing that we get back plain text anyway we don't need lynx. Also the sed-part removes the credit line.
This is sample output - yours may be different.
7

wget -qO - http://www.commandlinefu.com/commands/random/plaintext | sed -n '1d; /./p'

dramaturg · 2010-12-05 15:32:14 11
Look for English words in /dev/urandom

Little faster alternative.
This is sample output - yours may be different.
7

head -100000 /dev/urandom | strings|tr '[A-Z]' '[a-z]'|sort >temp.txt && wget -q http://www.mavi1.org/web_security/wordlists/webster-dictionary.txt -O-|tr '[A-Z]' '[a-z]'|sort >temp2.txt&&comm -12 temp.txt temp2.txt

marssi · 2010-12-26 11:04:42 6
Download from Rapidshare Premium using wget - Part 2

The download content part. NOTE: the '-c' seems to not work very well and the download stuck at 99% sometimes. Just finish wget with no problem. Also, the download may restart after complete. You can also cancel. I don't know if it is a wget or Rapidshare glitch since I don't have problems with Megaupload, for example. UPDATE: as pointed by roebek the restart glitch can be solved by the "-t 1" option. Thanks a lot.
This is sample output - yours may be different.
6

wget -c -t 1 --load-cookies ~/.cookies/rapidshare <URL>

o6291408 · 2009-03-28 09:13:35 5
background a wget download
This is sample output - yours may be different.
6

wget -b http://dl.google.com/android/android-sdk_r14-linux.tgz

zlemini · 2011-10-19 21:55:25 5
Find out how old a web page is

I used to use the Firefox "View page info" feature a lot to determine how stale the web page I was looking at was. Now that I use mostly Chrome I miss that feature, so here is a command line alternative using wget. The -S says to display the server response, the --spider says to not download any files/pages, just fetch the header. The output goes to stderr, so to grep it you use 2>&1 to combine the stderr stream with stdout, the pipe that to grep for Last-Modified. You can use curl instead if you have it installed, like this: curl --head -s http://osswin.sourceforge.net | grep Mod Show Sample Output
This is sample output - yours may be different.
```
  Last-Modified: Wed, 08 Feb 2012 20:14:53 GMT
```
6

wget -S --spider http://osswin.sourceforge.net/ 2>&1 | grep Mod

dmmst19 · 2012-04-18 03:43:33 6
download 10 random wallpapers from google

you may want &hl=en for &hl=es for the language you may want imgsz=xxlarge for imgsz=large or whatever filter you may want q=apples or whatever
This is sample output - yours may be different.
6

for i in {1..10};do wget $(wget -O- -U "" "http://images.google.com/images?imgsz=xxlarge&hl=en&q=wallpaper+HD&start=$(($RANDOM%900+100))" --quiet | grep -oe 'http://[^"]*\.jpg' | head -1);done

dzup · 2012-07-26 10:42:13 6
Update your OpenDNS network ip

Intended for dynamic ip OpenDNS users, this command will update your OpenDNS network IP. For getting your IP, you can use one of the many one-liners here on commandlinefu. Example: I use this in a script which is run by kppp after it has successfully connected to my ISP: --- #!/bin/bash IP="`curl -s http://checkip.dyndns.org/ | grep -o '[[:digit:].]\+'`" PW="hex-obfuscated-pw-here" if [ "$IP" == "" ] ; then echo 'Not online.' ; exit 1 else wget -q --user=topsecret --password="`echo $PW | xxd -ps -r`" 'https://updates.opendns.com/nic/update?hostname=myhostname&myip='"$IP" -O - /etc/init.d/ntp-client restart & fi --- PS: DynDNS should use a similar method, if you know the URL, please post a comment. (Something with members.dyndns.org, if I recall correctly) Show Sample Output
This is sample output - yours may be different.
```
alanceil@kvirasim:19:55:0:~> wget -q --user=topsecret --password=topsecret 'https://updates.opendns.com/nic/update?hostname=myhostname&myip=12.34.56.78' -O -
good 12.34.56.78
```
5

wget -q --user=<username> --password=<password> 'https://updates.opendns.com/nic/update?hostname=your_opendns_hostname&myip=your_ip' -O -

Alanceil · 2009-06-22 18:08:42 24
use wget to check if a remote file exists
This is sample output - yours may be different.
5

wget --spider -v http://www.server.com/path/file.ext

loetje · 2009-08-04 22:27:54 4
return external ip

I dont have curl or links installed, so I use wget with write file as standard out. Show Sample Output
This is sample output - yours may be different.
```
150.162.64.156
```
5

wget -O - -q icanhazip.com

andrepuel · 2009-08-14 10:24:30 14
Download Apple movie trailers

Copy the link to an HD movie trailer in to this command. It's more eleganant if it's put in a to a script, taking the URL as input.
This is sample output - yours may be different.
5

wget -U "QuickTime/7.6.2 (qtver=7.6.2;os=Windows NT 5.1Service Pack 3)" `echo http://movies.apple.com/movies/someHDmovie_720p.mov | sed 's/$[0-9][0-9]$0p/h\10p/'`

deadrabbit · 2009-08-29 00:29:40 21
download file1 file2 file3 file4 .... file 100
This is sample output - yours may be different.
5

wget http://domain.com/file{1..100}

wincus · 2010-01-12 14:28:04 3
Pronounce an English word using Merriam-Webster.com

The original was a little bit too complicated for me. This one does not use any variables.
This is sample output - yours may be different.
5

pronounce(){ wget -qO- $(wget -qO- "http://www.m-w.com/dictionary/$@" | grep 'return au' | sed -r "s|.*return au$'([^']*)', '([^'])[^']*'$.*|http://cougar.eb.com/soundc11/\2/\1|") | aplay -q; }

matthewbauer · 2010-03-12 17:44:16 6
translate what is in the clipboard in english and write it to the terminal

Uses google api to translate, you can modify the language in which translate modifying the parameter "langpair=|en", the format is language input|language output.
This is sample output - yours may be different.
5

wget -qO - "http://ajax.googleapis.com/ajax/services/language/translate?langpair=|en&v=1.0&q=`xsel`" |cut -d \" -f 6

fain182 · 2010-06-04 17:20:17 11
Google Translate

substitute "example" with desired string; tl = target language (en, fr, de, hu, ...); you can leave sl parameter as-is (autodetection works fine) Show Sample Output
This is sample output - yours may be different.
```
exemple de traduction
```
5

wget -U "Mozilla/5.0" -qO - "http://translate.google.com/translate_a/t?client=t&text=translation+example&sl=auto&tl=fr" | sed 's/\[\[\[\"//' | cut -d \" -f 1

sairon · 2011-03-06 13:46:16 109
Retrieve the size of a file on a server

- Where $URL is the URL of the file. - Replace the $2 by $3 at the end to get a human-readable size. Credits to svanberg @ ArchLinux forums for original idea. Edit: Replaced command with better version by FRUiT. (removed unnecessary grep)
This is sample output - yours may be different.
5

wget --spider $URL 2>&1 | awk '/Length/ {print $2}'

d3Xt3r · 2011-07-03 00:14:58 10
< 1 2 3 4 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Print all lines between two line numbers

Print all lines between two line numbers This command uses sed(1) to print all lines between two known line numbers in a file. Useful for seeing output in a log file, where the line numbers are known. The above command will print all lines between, and including, lines 3 and 6.

Output files without comments or empty lines

better integration. works on all Unices works one bash and ksh.

Scale,Rotate, brightness, contrast,...with Image Magick

$rotate: the rotate angle $width, $height: width and height to scale to $birghtness: change brighness

diff the same file in two directories.

This is useful when you're diffing two files of the same name in radically different directory trees. For example: Set $ path1='/some/long/convoluted/path/to/all/of/your/source/from/a/long/dead/machine' then $ path2='/local/version/of/same/file' then run the command. Much easier on the eyes when you're looking back across your command history, especially if you're doing the same diff over and over again.

Use Kernighan & Ritchie coding style in C program

Mount Fat USB with RWX

after that, you can launch bash script in your usb drive in FAT32.

Update zone file Serial numbers

Will edit *.db files in the same directory with todays date. Useful for doing a mass update to domains on a nameserver, adding spf records, etc. Looks for a string starting with 200 or 201 followed by 7 numbers, and replaces with todays date. This won't overwrite Ip's but i would still do some double checking after running this. Make sure your server's date is correct, otherwise insert your own serial number. $rndc reload should usually follow this command.

Rsync remote data as root using sudo

If your user has sudo on the remote box, you can rsync data as root without needing to login as root. This is very helpful if the remote box does not allow root to login over SSH (which is a common security restriction).

check open ports without netstat or lsof

Generate password

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using wget

Commands using wget (286) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using wget (286)

sorted by