Commands using wget (286)
-
0wget http://cmyip.com -O - -o /dev/null | grep -Po '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'
-
On a machine behind a firewall, it's possible to pass the proxy server address in as a prefix to wget to avoid having to set it as an environment variable first.
0http_proxy=<proxy.server:port> wget <url> -
This will visit recursively all linked urls starting from the specified URL. It won't save anything locally and it will produce a detailed log. Useful to find broken links in your site. It ignores robots.txt, so just use it on a site you own!
0wget --spider -o wget.log -e robots=off --wait 1 -r -p http://www.example.com -
This wgets the iso directly to the USB device, replace /dev/sdb with the device name of the USB stick. After wget finishes you will be able to boot the .iso file from the USB stick.
0wget -O/dev/sdb ftp://ftp.debian.org/debian/dists/stable/main/installer-amd64/current/images/netboot/mini.iso -
0for i in `seq -w 1 50`; do wget --continue \ http://commandline.org.uk/images/posts/animal/$i.jpg; done
-
No need to install additional packages eg: say hello For multiword say how+are+you
0say() { wget -q -U Mozilla -O output.mp3 "http://translate.google.com/translate_tts?tl=en&q=$1"; gnome-terminal -x bash -c "totem output.mp3"; sleep 4; totem --quit;} -
0curl -sm1 http://www.website.com/ | grep -o 'http://[^"]*jpg' | sort -u | wget -qT1 -i-
-
Recursively download all files of a certain type down to two levels, ignoring directory structure and local duplicates. Usage: wgetall mp3 http://example.com/download/
0wgetall () { wget -r -l2 -nd -Nc -A.$@ $@ } -
grabs and prints the AmazonMP3 daily album deal
0wget -qO- "http://www.amazon.com/b?ie=UTF8&node=163856011" | grep Daily | sed -e 's/<[^>]*>//g' -e 's/^ *//' -e 's/\&[^;]*;/ /' -
Encrypt any text to MD5 , replace text with the string you want to convert to MD5 Show Sample Output
0wget -qO - --post-data "data[Row][clear]=text" http://md5-encryption.com | grep -A1 "Md5 encrypted state" | tail -n1 | cut -d '"' -f3 | sed 's/>//g; s/<\/b//g' -
This shell function uses wget(1) to show what site a shortened URL is pointing to, even if there are many nested shortened URLs. This is a great way to test whether or not the shortened URL is sending you to a malicious site, or somewhere nasty that you don't want to visit. The sample output is from:
expandurl http://t.co/LDWqmtDMShow Sample Output0expandurl() { wget -S $1 2>&1 | grep ^Location; } -
Uses wget standard GNU utility. Prints only your ip.
0wget -qO - http://whatismyip.org | tail -
Doesn't depend on curl and doesn't use thumbnails as wallpaper (which has the unfortunate effect of only allowing imgur links)
0wget -O - http://www.reddit.com/r/wallpapers.rss | grep -Eo 'http://i.imgur.com[^&]+jpg' | head -1 | xargs wget -O background.jpg -
0wget --no-use-server-timestamps $(curl $(curl http://wallbase.cc/random/23/eqeq/1920x1080/0/100/20 | grep 'wallpaper/' | awk -F'"' '{print $2}' | head -n1) | grep -A4 bigwall | grep img | awk -F'"' '{print $2}'); feh --bg-center $(ls -1t | head -n1)
-
0wget -mk -w 20 http://www.example.com/
-
Tries to avoid the fragile nature of scrapers by looking for user-input in the output as opposed to markup or headers on the web site. Show Sample Output
0function ip-where { wget -qO- -U Mozilla http://www.ip-adress.com/ip_tracer/$1 | html2text -nobs -style pretty | sed -n /^$1/,/^$/p;} -
0wget --recursive --page-requisites --convert-links www.moyagraphix.co.za
-
Very similar as doing "wget http://example.com/mytarball|tar xzv", this one involves the "tee" command between both, which will simultaneously write the tarball and copy it to stdout. So this command will locally save the tarball and extract it - both at the same time while it downloads.
0wget -O- http://example.com/mytarball.tgz | tee mytarball.tgz | tar xzv -
0wget -O - "[PICASA ALBUM RSS LINK]" |sed 's/</\n</g' | grep media:content |sed 's/.*url='"'"'\([^'"'"']*\)'"'"'.*$/\1/' |awk -F'/' '{gsub($NF,"d/"$NF); print $0}'|wget -i -
-
Reason can be: taken, available, contains_banned_word
0if wget https://twitter.com/users/username_available?username=xmuda -q -O - | grep -q "\"reason\":\"taken\""; then echo "Username taken"; else echo "Free / Banned Name"; fi -
remove old index.html if you download it again and organiaz the java script tag on the file index.html
0rm index.html | wget www.google.com;cat index.html | sed 's/<script>/\n\n\<script>\n\n/g' | sed 's/<\/script>/>\n\n/g' -
This command will help you to get Ip address origin of city and country. I will be happy if someone can shrink the sed command fb.me/josenirmal Show Sample Output
0wget -qO - whatismyipaddress.com/ip/<type ip address> | grep -E "City:|Country:" | sed 's:<tr><th>::'| sed 's</th>::' | sed 's:</td>::' | sed 's:</tr>::' | sed 's:<img*::' -
0site=https://www.duden.de; wort="Apfel"; wget -O $wort.mp3 $(wget -O - "$site/rechtschreibung/$wort" | grep -o "$site/_media_/audio/[^\.]*\.mp3")
-
This script can be used to download enclosed files from a RSS feed. For example, it can be used to download mp3 files from a podcasts RSS feed. Show Sample Output
0wget -q -O- http://example-podcast-feed.com/rss | grep -o "<enclosure[ -~][^>]*" | grep -o "http://[ -~][^\"]*" | xargs wget -c -
Access www.kernel.org and download the last stable linux kernel release. Show Sample Output
0wget --no-check-certificate https://www.kernel.org/$(wget -qO- --no-check-certificate https://www.kernel.org | grep tar | head -n1 | cut -d\" -f2) - ‹ First < 6 7 8 9 10 > Last ›
What's this?
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Check These Out
Stay in the loop…
- Follow the Tweets.
-
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10 - Subscribe to the feeds.
-
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
- » all commands
- » commands with 3 up-votes (commandlinefu3)
- » commands with 10 up-votes (commandlinefu10)
- » commands using wget