Commands using wget sorted by votes

Commands using wget (286)

sorted by

Wget Command to Download Full Recursive Version of Web Page

The example will create a directory called " Caveats: @imports of css files will not be converted.
This is sample output - yours may be different.
8

wget -p --convert-links http://www.foo.com

damniel · 2009-02-13 19:02:08 11
Poke a Webserver to see what it's powered by.

the good: Server: Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.4 with Suhosin-Patch the bad: Server: Microsoft-IIS/6.0 and the ugly: Server: Apache/2.2.10 (Win32) mod_ssl/2.2.10 OpenSSL/0.9.8i PHP/5.2.6
This is sample output - yours may be different.
8

wget -S -O/dev/null "INSERT_URL_HERE" 2>&1 | grep Server

asmoore82 · 2009-03-09 06:54:54 7
Download free e-books

Mask the user agent as firefox, recursively download 2 levels deep from a span host with a maximum of 1 redirection, use random wait time and dump all pdf files to myBooksFolder without creating any other directories. Host will have no way of knowing that this is a grabber script.
This is sample output - yours may be different.
8

wget -erobots=off --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3" -H -r -l2 --max-redirect=1 -w 5 --random-wait -PmyBooksFolder -nd --no-parent -A.pdf http://URL

unixmonkey8504 · 2010-03-01 20:22:44 29
Google Translate

allow multiword translations Show Sample Output
This is sample output - yours may be different.
```
$ translate en de I am looking for a table
```
8

translate() { lng1="$1";lng2="$2";shift;shift; wget -qO- "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=${@// /+}&langpair=$lng1|$lng2" | sed 's/.*"translatedText":"$[^"]*$".*}/\1\n/'; }

bandie91 · 2010-05-02 22:15:30 36
Get all files of particular type (say, PDF) listed on some wegpage (say, example.com)

See man wget if you want linked files and not only those hosted on the website.
This is sample output - yours may be different.
8

wget -r -A .pdf -l 5 -nH --no-parent http://example.com

houghi · 2011-06-09 17:17:03 6
Download an entire ftp directory using wget

If the username includes an @ you can use this one: wget -r --user=username_here --password=pass_here ftp://ftp.example.com
This is sample output - yours may be different.
7

wget -r ftp://user:pass@ftp.example.com

movaxes · 2009-03-09 19:39:30 13
Dump HTTP header using wget

Let me suggest using wget for obtaining the HTTP header only as the last resort because it generates considerable textual overhead. The first ellipsis of the sample output stands for Spider mode enabled. Check if remote file exists. --2009-03-31 20:42:46-- http://www.example.com/ Resolving www.example.com... 208.77.188.166 Connecting to www.example.com|208.77.188.166|:80... connected. HTTP request sent, awaiting response... and the second one looks for Length: 438 [text/html] Remote file exists and could contain further links, but recursion is disabled -- not retrieving. Show Sample Output
This is sample output - yours may be different.
```
&#8230;
  HTTP/1.1 200 OK
  Date: Tue, 31 Mar 2009 18:42:46 GMT
  Server: Apache/2.2.3 (CentOS)
  Last-Modified: Tue, 15 Nov 2005 13:24:10 GMT
  ETag: "b80f4-1b6-80bfd280"
  Accept-Ranges: bytes
  Content-Length: 438
  Connection: close
  Content-Type: text/html; charset=UTF-8
&#8230;
```
7

wget --server-response --spider http://www.example.com/

penpen · 2009-03-31 18:49:14 15
Block the 6700 worst spamhosts

The above url contains over 6700 of the common ad websites. The command just pastes these into your /etc/hosts. Show Sample Output
This is sample output - yours may be different.
```
-
```
7

wget -q -O - http://someonewhocares.org/hosts/ | grep ^127 >> /etc/hosts

torrid · 2009-12-14 17:11:16 8
Limit bandwidth usage by any program

Trickle is a voluntary, cooperative bandwidth shaper. it works entirely in userland and is very easy to use. The most simple application is to limit the bandwidth usage of programs.
This is sample output - yours may be different.
7

trickle -d 60 wget http://very.big/file

point_to_null · 2010-03-29 06:55:30 5
Lets Tux say the random fact. [add it to .bashrc to see it in new terminal window]
This is sample output - yours may be different.
7

wget randomfunfacts.com -O - 2>/dev/null|grep \<strong\>|sed "s;^.*<i>$.*$</i>.*$;\1;"|cowsay -f tux

unixmonkey10612 · 2010-07-01 18:31:36 4
Retrieve a random command from the commandlinefu.com API

Seeing that we get back plain text anyway we don't need lynx. Also the sed-part removes the credit line.
This is sample output - yours may be different.
7

wget -qO - http://www.commandlinefu.com/commands/random/plaintext | sed -n '1d; /./p'

dramaturg · 2010-12-05 15:32:14 11
Look for English words in /dev/urandom

Little faster alternative.
This is sample output - yours may be different.
7

head -100000 /dev/urandom | strings|tr '[A-Z]' '[a-z]'|sort >temp.txt && wget -q http://www.mavi1.org/web_security/wordlists/webster-dictionary.txt -O-|tr '[A-Z]' '[a-z]'|sort >temp2.txt&&comm -12 temp.txt temp2.txt

marssi · 2010-12-26 11:04:42 6
Download from Rapidshare Premium using wget - Part 2

The download content part. NOTE: the '-c' seems to not work very well and the download stuck at 99% sometimes. Just finish wget with no problem. Also, the download may restart after complete. You can also cancel. I don't know if it is a wget or Rapidshare glitch since I don't have problems with Megaupload, for example. UPDATE: as pointed by roebek the restart glitch can be solved by the "-t 1" option. Thanks a lot.
This is sample output - yours may be different.
6

wget -c -t 1 --load-cookies ~/.cookies/rapidshare <URL>

o6291408 · 2009-03-28 09:13:35 5
background a wget download
This is sample output - yours may be different.
6

wget -b http://dl.google.com/android/android-sdk_r14-linux.tgz

zlemini · 2011-10-19 21:55:25 5
Find out how old a web page is

I used to use the Firefox "View page info" feature a lot to determine how stale the web page I was looking at was. Now that I use mostly Chrome I miss that feature, so here is a command line alternative using wget. The -S says to display the server response, the --spider says to not download any files/pages, just fetch the header. The output goes to stderr, so to grep it you use 2>&1 to combine the stderr stream with stdout, the pipe that to grep for Last-Modified. You can use curl instead if you have it installed, like this: curl --head -s http://osswin.sourceforge.net | grep Mod Show Sample Output
This is sample output - yours may be different.
```
  Last-Modified: Wed, 08 Feb 2012 20:14:53 GMT
```
6

wget -S --spider http://osswin.sourceforge.net/ 2>&1 | grep Mod

dmmst19 · 2012-04-18 03:43:33 6
download 10 random wallpapers from google

you may want &hl=en for &hl=es for the language you may want imgsz=xxlarge for imgsz=large or whatever filter you may want q=apples or whatever
This is sample output - yours may be different.
6

for i in {1..10};do wget $(wget -O- -U "" "http://images.google.com/images?imgsz=xxlarge&hl=en&q=wallpaper+HD&start=$(($RANDOM%900+100))" --quiet | grep -oe 'http://[^"]*\.jpg' | head -1);done

dzup · 2012-07-26 10:42:13 6
Update your OpenDNS network ip

Intended for dynamic ip OpenDNS users, this command will update your OpenDNS network IP. For getting your IP, you can use one of the many one-liners here on commandlinefu. Example: I use this in a script which is run by kppp after it has successfully connected to my ISP: --- #!/bin/bash IP="`curl -s http://checkip.dyndns.org/ | grep -o '[[:digit:].]\+'`" PW="hex-obfuscated-pw-here" if [ "$IP" == "" ] ; then echo 'Not online.' ; exit 1 else wget -q --user=topsecret --password="`echo $PW | xxd -ps -r`" 'https://updates.opendns.com/nic/update?hostname=myhostname&myip='"$IP" -O - /etc/init.d/ntp-client restart & fi --- PS: DynDNS should use a similar method, if you know the URL, please post a comment. (Something with members.dyndns.org, if I recall correctly) Show Sample Output
This is sample output - yours may be different.
```
alanceil@kvirasim:19:55:0:~> wget -q --user=topsecret --password=topsecret 'https://updates.opendns.com/nic/update?hostname=myhostname&myip=12.34.56.78' -O -
good 12.34.56.78
```
5

wget -q --user=<username> --password=<password> 'https://updates.opendns.com/nic/update?hostname=your_opendns_hostname&myip=your_ip' -O -

Alanceil · 2009-06-22 18:08:42 24
use wget to check if a remote file exists
This is sample output - yours may be different.
5

wget --spider -v http://www.server.com/path/file.ext

loetje · 2009-08-04 22:27:54 4
return external ip

I dont have curl or links installed, so I use wget with write file as standard out. Show Sample Output
This is sample output - yours may be different.
```
150.162.64.156
```
5

wget -O - -q icanhazip.com

andrepuel · 2009-08-14 10:24:30 14
Download Apple movie trailers

Copy the link to an HD movie trailer in to this command. It's more eleganant if it's put in a to a script, taking the URL as input.
This is sample output - yours may be different.
5

wget -U "QuickTime/7.6.2 (qtver=7.6.2;os=Windows NT 5.1Service Pack 3)" `echo http://movies.apple.com/movies/someHDmovie_720p.mov | sed 's/$[0-9][0-9]$0p/h\10p/'`

deadrabbit · 2009-08-29 00:29:40 21
download file1 file2 file3 file4 .... file 100
This is sample output - yours may be different.
5

wget http://domain.com/file{1..100}

wincus · 2010-01-12 14:28:04 3
Pronounce an English word using Merriam-Webster.com

The original was a little bit too complicated for me. This one does not use any variables.
This is sample output - yours may be different.
5

pronounce(){ wget -qO- $(wget -qO- "http://www.m-w.com/dictionary/$@" | grep 'return au' | sed -r "s|.*return au$'([^']*)', '([^'])[^']*'$.*|http://cougar.eb.com/soundc11/\2/\1|") | aplay -q; }

matthewbauer · 2010-03-12 17:44:16 6
translate what is in the clipboard in english and write it to the terminal

Uses google api to translate, you can modify the language in which translate modifying the parameter "langpair=|en", the format is language input|language output.
This is sample output - yours may be different.
5

wget -qO - "http://ajax.googleapis.com/ajax/services/language/translate?langpair=|en&v=1.0&q=`xsel`" |cut -d \" -f 6

fain182 · 2010-06-04 17:20:17 11
Google Translate

substitute "example" with desired string; tl = target language (en, fr, de, hu, ...); you can leave sl parameter as-is (autodetection works fine) Show Sample Output
This is sample output - yours may be different.
```
exemple de traduction
```
5

wget -U "Mozilla/5.0" -qO - "http://translate.google.com/translate_a/t?client=t&text=translation+example&sl=auto&tl=fr" | sed 's/\[\[\[\"//' | cut -d \" -f 1

sairon · 2011-03-06 13:46:16 109
Retrieve the size of a file on a server

- Where $URL is the URL of the file. - Replace the $2 by $3 at the end to get a human-readable size. Credits to svanberg @ ArchLinux forums for original idea. Edit: Replaced command with better version by FRUiT. (removed unnecessary grep)
This is sample output - yours may be different.
5

wget --spider $URL 2>&1 | awk '/Length/ {print $2}'

d3Xt3r · 2011-07-03 00:14:58 10
< 1 2 3 4 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

To find which host made maximum number of specific tcp connections

This command is primarily going to work on linux boxes. and needs to be changed, for example IP=10\.194\.194\.2 PORT=389

processes per user counter

enumerates the number of processes for each user. ps BSD format is used here , for standard Unix format use : ps -eLf |awk '{$1} {++P[$1]} END {for(a in P) if (a !="UID") print a,P[a]}'

gets your public IP address

Text graphing ping output filter

Nasty perl one-liner that provides a sparkline of ping times. If you want a different history than the last 30, just put that value in. It (ab)uses unicode to draw the bars, inspired by https://github.com/joemiller/spark-ping . It's not the most bug-free piece of code, but what it lacks in robustness it makes up for in capability. :) If anyone has any ideas on how to make it more compact or better, I'd love to hear them. I included a ping to google in the command just as an example (and burned up 10 chars doing it!). You should use it with: $ ping example.com | $SPARKLINE_PING_COMMAND

The program listening on port 8080 through IPv6

Random Number Between 1 And X

If X is 5, it will about a number between 1 and 5 inclusive. This works in bash and zsh. If you want between 0 and 4, remove the +1.

Empty a file

Renames all files in the current directory such that the new file contains no space characters.

This is a better version, as it does no command piping, uses for instead of while loops, which allows for a list of files in the current working directory to be natively processed. It also uses the -v/verbose option with mv to let you know what the command is doing. While the command does exactly the same in a better way, I would modify the sed option to replace spaces with underscores instead, or dashes. Please note that you'll receive errors with this command as it tries to rename files that don't even have spaces. This is an alternative to: http://www.commandlinefu.com/commands/view/8761/renames-all-files-in-the-current-directory-such-that-the-new-file-contains-no-space-characters.

print all except first collumn

Convert all JPEG images to MP4

Source: http://superuser.com/a/624574

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using wget

Commands using wget (286) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using wget (286)

sorted by