Commands tagged wget

This example command fetches 'example.com' webpage and then fetches+saves all PDF files listed (linked to) on that webpage.

[*Note: of course there are no PDFs on example.com. This is just an example]

This will visit recursively all linked urls starting from the specified URL. It won't save anything locally and it will produce a detailed log.

Useful to find broken links in your site. It ignores robots.txt, so just use it on a site you own!

On a machine behind a firewall, it's possible to pass the proxy server address in as a prefix to wget to avoid having to set it as an environment variable first.

squeezed the monster (and nifty ☺) command from 7776 from 531 characters to 284 characters, but I don't see a way to get it down to 255. This is definitely a kludge!

Substitute that 724349691704 with an UPC of a CD you have at hand, and (hopefully) this oneliner should return the $Artist - $Title, querying discogs.com.

Yes, I know, all that head/tail/grep crap can be improved with a single sed command, feel free to send "patches" :D

Enjoy!

Yep, now you can finally google from the command line!

Here's a readable version "for your pleasure"(c):

Enjoy :)

To learn more about Google Ngram Viewer: http://ngrams.googlelabs.com/info

Get gzip compressed web page using wget.

Caution: The command will fail in case website doesn't return gzip encoded content, though most of thw websites have gzip support now a days.

Just added view with the eog viewer.

The command was too long for the command box, so here it is:

This took me about an hour to do. It uses wget and curl because, dudalibre.com blocks wget, and wget worked nicely for me.

I use this command in my Conky script to display the number of messages in my Gmail inbox and to list the from: and subject: fields.

The "-k" flag will tell wget to convert links for local browsing; it works with mirroring (ie with "-r") or single-file downloads.

extension to tali713's random fact generator. It takes the output & sends it to notify-osd. Display time is proportional to the lengh of the fact.

Though without infinite time and knowledge of how the site will be designed in the future this may stop working, it still will serve as a simple straight forward starting point.

This uses the observation that the only item marked as strong on the page is the single logical line that includes the italicized fact.

If future revisions of the page show failure, or intermittent failure, one may simply alter the above to read.

The file lastfact, can then be examined whenever the command fails.

Bash scrip to test if a server is up, you can use this before wget'ing a file to make sure a blank one isn't downloaded.

This one uses dictionary.com

The original was a little bit too complicated for me. This one does not use any variables.

Looks up a word on merriam-webster.com, does a screen scrape for the FIRST audio pronunciation and plays it.

USAGE: Put this one-liner into a shell script (e.g., ~/bin/pronounce) and run it from the command line giving it the word to say:

If the word isn't found in merriam-webster, no audio is played and the script returns an error value. However, M-W is a fairly complete dictionary (better than howjsay.com which won't let you hear how to pronounce naughty words).

ASSUMPTIONS: GNU's sed (which supports -r for extended regular expressions) and Linux's aplay. Aplay can be replaced by any program that can play .WAV files from stdin.

KNOWN BUGS: only the FIRST pronunciation is played, which is problematic if you wanted a particular form (plural, adjectival, etc) of the word. For example, if you run this:

you'll hear a voice saying "onomatopoeia".

Playing the correct form of the word is possible, but doing so might make the screen scraper even more fragile than it already is. (The slightest change to the format of m-w.com could break it).

This will send the web page at $u to recipient@example.com . To send the web page to oneself, recipient@example.com can be replaced by $(whoami) .

The "charset" is UTF-8 here, but any alternative charset of your choice would work.

`wget -O - -o /dev/null $u` may be considered instead of `curl $u` .

On some systems the complete path to sendmail may be necessary, for instance /sys/pkg/libexec/sendmail/sendmail for some NetBSD.

Prompts the user for username and password, that are then exported to http_proxy for use by wget, yum etc

Default user, webproxy and port are used.

Using this script prevent the cleartext user and pass being in your bash_history and on-screen

This is a minimalistic version of the ubiquitious Google definition screen scraper. This version was designed not only to run fast, but to work using BusyBox. BusyBox is a collection of basic Unix tools that have been compiled into a single binary to save space on tiny installations of Unix. For example, although my phone doesn't have perl or the GNU utilities, it does have BusyBox's stripped down versions of wget, tr, and sed. It turns out that those tools suffice for many tasks.

Known Bugs: This script does not handle HTML entities at all. I don't think there's an easy way to do that within BusyBox, but I'd love to see it if someone could do it. Also, this script can only define a single word, not phrases. (Well, you could if you typed in %20, but that'd be gross.) Lastly, this script does not show the URL where definitions were found. Given the randomness of the Net, that last bit of information is often key.

Only need to install Image Magick package.

Display a xkcd comic with its title and save it in /tmp directory

If you prefer to view the newest xkcd, use this command: