Commands using wget sorted by votes

Commands using wget (286)

sorted by

Wget Command to Download Full Recursive Version of Web Page

The example will create a directory called " Caveats: @imports of css files will not be converted.
This is sample output - yours may be different.
8

wget -p --convert-links http://www.foo.com

damniel · 2009-02-13 19:02:08 11
Poke a Webserver to see what it's powered by.

the good: Server: Apache/2.2.8 (Ubuntu) PHP/5.2.4-2ubuntu5.4 with Suhosin-Patch the bad: Server: Microsoft-IIS/6.0 and the ugly: Server: Apache/2.2.10 (Win32) mod_ssl/2.2.10 OpenSSL/0.9.8i PHP/5.2.6
This is sample output - yours may be different.
8

wget -S -O/dev/null "INSERT_URL_HERE" 2>&1 | grep Server

asmoore82 · 2009-03-09 06:54:54 7
Download free e-books

Mask the user agent as firefox, recursively download 2 levels deep from a span host with a maximum of 1 redirection, use random wait time and dump all pdf files to myBooksFolder without creating any other directories. Host will have no way of knowing that this is a grabber script.
This is sample output - yours may be different.
8

wget -erobots=off --user-agent="Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.3) Gecko/2008092416 Firefox/3.0.3" -H -r -l2 --max-redirect=1 -w 5 --random-wait -PmyBooksFolder -nd --no-parent -A.pdf http://URL

unixmonkey8504 · 2010-03-01 20:22:44 29
Google Translate

allow multiword translations Show Sample Output
This is sample output - yours may be different.
```
$ translate en de I am looking for a table
```
8

translate() { lng1="$1";lng2="$2";shift;shift; wget -qO- "http://ajax.googleapis.com/ajax/services/language/translate?v=1.0&q=${@// /+}&langpair=$lng1|$lng2" | sed 's/.*"translatedText":"$[^"]*$".*}/\1\n/'; }

bandie91 · 2010-05-02 22:15:30 36
Get all files of particular type (say, PDF) listed on some wegpage (say, example.com)

See man wget if you want linked files and not only those hosted on the website.
This is sample output - yours may be different.
8

wget -r -A .pdf -l 5 -nH --no-parent http://example.com

houghi · 2011-06-09 17:17:03 6
Download an entire ftp directory using wget

If the username includes an @ you can use this one: wget -r --user=username_here --password=pass_here ftp://ftp.example.com
This is sample output - yours may be different.
7

wget -r ftp://user:pass@ftp.example.com

movaxes · 2009-03-09 19:39:30 13
Dump HTTP header using wget

Let me suggest using wget for obtaining the HTTP header only as the last resort because it generates considerable textual overhead. The first ellipsis of the sample output stands for Spider mode enabled. Check if remote file exists. --2009-03-31 20:42:46-- http://www.example.com/ Resolving www.example.com... 208.77.188.166 Connecting to www.example.com|208.77.188.166|:80... connected. HTTP request sent, awaiting response... and the second one looks for Length: 438 [text/html] Remote file exists and could contain further links, but recursion is disabled -- not retrieving. Show Sample Output
This is sample output - yours may be different.
```
&#8230;
  HTTP/1.1 200 OK
  Date: Tue, 31 Mar 2009 18:42:46 GMT
  Server: Apache/2.2.3 (CentOS)
  Last-Modified: Tue, 15 Nov 2005 13:24:10 GMT
  ETag: "b80f4-1b6-80bfd280"
  Accept-Ranges: bytes
  Content-Length: 438
  Connection: close
  Content-Type: text/html; charset=UTF-8
&#8230;
```
7

wget --server-response --spider http://www.example.com/

penpen · 2009-03-31 18:49:14 15
Block the 6700 worst spamhosts

The above url contains over 6700 of the common ad websites. The command just pastes these into your /etc/hosts. Show Sample Output
This is sample output - yours may be different.
```
-
```
7

wget -q -O - http://someonewhocares.org/hosts/ | grep ^127 >> /etc/hosts

torrid · 2009-12-14 17:11:16 8
Limit bandwidth usage by any program

Trickle is a voluntary, cooperative bandwidth shaper. it works entirely in userland and is very easy to use. The most simple application is to limit the bandwidth usage of programs.
This is sample output - yours may be different.
7

trickle -d 60 wget http://very.big/file

point_to_null · 2010-03-29 06:55:30 5
Lets Tux say the random fact. [add it to .bashrc to see it in new terminal window]
This is sample output - yours may be different.
7

wget randomfunfacts.com -O - 2>/dev/null|grep \<strong\>|sed "s;^.*<i>$.*$</i>.*$;\1;"|cowsay -f tux

unixmonkey10612 · 2010-07-01 18:31:36 3
Retrieve a random command from the commandlinefu.com API

Seeing that we get back plain text anyway we don't need lynx. Also the sed-part removes the credit line.
This is sample output - yours may be different.
7

wget -qO - http://www.commandlinefu.com/commands/random/plaintext | sed -n '1d; /./p'

dramaturg · 2010-12-05 15:32:14 11
Look for English words in /dev/urandom

Little faster alternative.
This is sample output - yours may be different.
7

head -100000 /dev/urandom | strings|tr '[A-Z]' '[a-z]'|sort >temp.txt && wget -q http://www.mavi1.org/web_security/wordlists/webster-dictionary.txt -O-|tr '[A-Z]' '[a-z]'|sort >temp2.txt&&comm -12 temp.txt temp2.txt

marssi · 2010-12-26 11:04:42 4
Download from Rapidshare Premium using wget - Part 2

The download content part. NOTE: the '-c' seems to not work very well and the download stuck at 99% sometimes. Just finish wget with no problem. Also, the download may restart after complete. You can also cancel. I don't know if it is a wget or Rapidshare glitch since I don't have problems with Megaupload, for example. UPDATE: as pointed by roebek the restart glitch can be solved by the "-t 1" option. Thanks a lot.
This is sample output - yours may be different.
6

wget -c -t 1 --load-cookies ~/.cookies/rapidshare <URL>

o6291408 · 2009-03-28 09:13:35 5
background a wget download
This is sample output - yours may be different.
6

wget -b http://dl.google.com/android/android-sdk_r14-linux.tgz

zlemini · 2011-10-19 21:55:25 5
Find out how old a web page is

I used to use the Firefox "View page info" feature a lot to determine how stale the web page I was looking at was. Now that I use mostly Chrome I miss that feature, so here is a command line alternative using wget. The -S says to display the server response, the --spider says to not download any files/pages, just fetch the header. The output goes to stderr, so to grep it you use 2>&1 to combine the stderr stream with stdout, the pipe that to grep for Last-Modified. You can use curl instead if you have it installed, like this: curl --head -s http://osswin.sourceforge.net | grep Mod Show Sample Output
This is sample output - yours may be different.
```
  Last-Modified: Wed, 08 Feb 2012 20:14:53 GMT
```
6

wget -S --spider http://osswin.sourceforge.net/ 2>&1 | grep Mod

dmmst19 · 2012-04-18 03:43:33 6
download 10 random wallpapers from google

you may want &hl=en for &hl=es for the language you may want imgsz=xxlarge for imgsz=large or whatever filter you may want q=apples or whatever
This is sample output - yours may be different.
6

for i in {1..10};do wget $(wget -O- -U "" "http://images.google.com/images?imgsz=xxlarge&hl=en&q=wallpaper+HD&start=$(($RANDOM%900+100))" --quiet | grep -oe 'http://[^"]*\.jpg' | head -1);done

dzup · 2012-07-26 10:42:13 6
Update your OpenDNS network ip

Intended for dynamic ip OpenDNS users, this command will update your OpenDNS network IP. For getting your IP, you can use one of the many one-liners here on commandlinefu. Example: I use this in a script which is run by kppp after it has successfully connected to my ISP: --- #!/bin/bash IP="`curl -s http://checkip.dyndns.org/ | grep -o '[[:digit:].]\+'`" PW="hex-obfuscated-pw-here" if [ "$IP" == "" ] ; then echo 'Not online.' ; exit 1 else wget -q --user=topsecret --password="`echo $PW | xxd -ps -r`" 'https://updates.opendns.com/nic/update?hostname=myhostname&myip='"$IP" -O - /etc/init.d/ntp-client restart & fi --- PS: DynDNS should use a similar method, if you know the URL, please post a comment. (Something with members.dyndns.org, if I recall correctly) Show Sample Output
This is sample output - yours may be different.
```
alanceil@kvirasim:19:55:0:~> wget -q --user=topsecret --password=topsecret 'https://updates.opendns.com/nic/update?hostname=myhostname&myip=12.34.56.78' -O -
good 12.34.56.78
```
5

wget -q --user=<username> --password=<password> 'https://updates.opendns.com/nic/update?hostname=your_opendns_hostname&myip=your_ip' -O -

Alanceil · 2009-06-22 18:08:42 24
use wget to check if a remote file exists
This is sample output - yours may be different.
5

wget --spider -v http://www.server.com/path/file.ext

loetje · 2009-08-04 22:27:54 4
return external ip

I dont have curl or links installed, so I use wget with write file as standard out. Show Sample Output
This is sample output - yours may be different.
```
150.162.64.156
```
5

wget -O - -q icanhazip.com

andrepuel · 2009-08-14 10:24:30 14
Download Apple movie trailers

Copy the link to an HD movie trailer in to this command. It's more eleganant if it's put in a to a script, taking the URL as input.
This is sample output - yours may be different.
5

wget -U "QuickTime/7.6.2 (qtver=7.6.2;os=Windows NT 5.1Service Pack 3)" `echo http://movies.apple.com/movies/someHDmovie_720p.mov | sed 's/$[0-9][0-9]$0p/h\10p/'`

deadrabbit · 2009-08-29 00:29:40 21
download file1 file2 file3 file4 .... file 100
This is sample output - yours may be different.
5

wget http://domain.com/file{1..100}

wincus · 2010-01-12 14:28:04 3
Pronounce an English word using Merriam-Webster.com

The original was a little bit too complicated for me. This one does not use any variables.
This is sample output - yours may be different.
5

pronounce(){ wget -qO- $(wget -qO- "http://www.m-w.com/dictionary/$@" | grep 'return au' | sed -r "s|.*return au$'([^']*)', '([^'])[^']*'$.*|http://cougar.eb.com/soundc11/\2/\1|") | aplay -q; }

matthewbauer · 2010-03-12 17:44:16 6
translate what is in the clipboard in english and write it to the terminal

Uses google api to translate, you can modify the language in which translate modifying the parameter "langpair=|en", the format is language input|language output.
This is sample output - yours may be different.
5

wget -qO - "http://ajax.googleapis.com/ajax/services/language/translate?langpair=|en&v=1.0&q=`xsel`" |cut -d \" -f 6

fain182 · 2010-06-04 17:20:17 11
Google Translate

substitute "example" with desired string; tl = target language (en, fr, de, hu, ...); you can leave sl parameter as-is (autodetection works fine) Show Sample Output
This is sample output - yours may be different.
```
exemple de traduction
```
5

wget -U "Mozilla/5.0" -qO - "http://translate.google.com/translate_a/t?client=t&text=translation+example&sl=auto&tl=fr" | sed 's/\[\[\[\"//' | cut -d \" -f 1

sairon · 2011-03-06 13:46:16 109
Retrieve the size of a file on a server

- Where $URL is the URL of the file. - Replace the $2 by $3 at the end to get a human-readable size. Credits to svanberg @ ArchLinux forums for original idea. Edit: Replaced command with better version by FRUiT. (removed unnecessary grep)
This is sample output - yours may be different.
5

wget --spider $URL 2>&1 | awk '/Length/ {print $2}'

d3Xt3r · 2011-07-03 00:14:58 10
< 1 2 3 4 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Convert seconds to [DD:][HH:]MM:SS

Converts any number of seconds into days, hours, minutes and seconds. sec2dhms() { declare -i SS="$1" D=$(( SS / 86400 )) H=$(( SS % 86400 / 3600 )) M=$(( SS % 3600 / 60 )) S=$(( SS % 60 )) [ "$D" -gt 0 ] && echo -n "${D}:" [ "$H" -gt 0 ] && printf "%02g:" "$H" printf "%02g:%02g\n" "$M" "$S" }

Show errors in the kernel ring buffer

Much more useful then parsing syslog

Compress and store the image of a disk over the network

Create an image of "device" and send it to another machine through the network ("target" and "port" sets the ip and port the stream will be sent to), outputting a progress bar On the machine that will receive, compress and store the file, use: $nc -l -p | 7z a -si -m0=lzma2 -mx=9 -ms=on Optionally, add the -v4g switch at the end of the line in order to split the file every 4 gigabytes (or set another size: accepted suffixes are k, m and g). The file will be compressed using 7z format, lzma2 algorithm, with maximum compression level and solid file activated. The compression stage will be executed on the machine which will store the image. It was planned this way because the processor on that machine was faster, and being on a gigabit network, transfering the uncompressed image wasn't much of a problem.

urldecoding with one pure BASH builtin

You can use ordinary printf to convert "%23%21%2fbin%2fbash" into "#!/bin/bash" with no external utilities, by using a little known printf feature -- the "%b" specifier converts shell escapes. Replace % with \x and printf will understand the urlencoded string. BASH's printf has an extension to set a variable directly, too. So you get to convert urlencoded strings from garble to plaintext in one step with no externals and no backticks.

need ascii art pictures for you readme text ?

Require boxes and / or cowsay packages. After input boxes -d dog , type your text and then press ctrl + d . Same goes for cowsay .

Define words and phrases with google.

This version works on Mac (avoids grep -P, adding a sed step instead, and invokes /usr/bin/perl with full path in case you have another one installed). Still requires that you install perl module HTML::Entities ? here's how: http://www.perlmonks.org/?node_id=640489

Alias to edit and source your .bashrc file

Place the line above in your ~/.bahsrc file. Now every time you issue the 'vb' command, you invoke the vim editor to edit it, then source it so the changes take effect immediately. Notes: * This mechanism is not working well if your .bashrc contains commands that should not be sourced more than once. * This trick also work for your csh or tclsh users: place the following line in your ~/.cshrc file: alias vc 'vim ~/.cshrc; source ~/.cshrc Thank you adzap for pointing out the missing quote

how many packages installed on your archlinux?

View a sopcast stream

Get sopcast links for live sports from http://myp2p.eu (for example) Get sp-sc or sp-auth binary by googling (sopcast + linux) eg http://www.jbg.f2s.com/sp-sc.gz Requires the 32bit libstdc++5 package. After exiting mplayer, type 'killall sp-sc'

Twitter update from terminal (pok3's snipts ?)

Found it on snipt, pok3, is it yours? I put my user = m33600, the password and the status was my robot message: Settima robot message: ALARM ZONE 3 (sent via command line). Now bots may have their identity on twitter...

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using wget

Commands using wget (286) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using wget (286)

sorted by