What happens here is we tell tar to create "-c" an archive of all files in current dir "." (recursively) and output the data to stdout "-f -". Next we specify the size "-s" to pv of all files in current dir. The "du -sb . | awk ?{print $1}?" returns number of bytes in current dir, and it gets fed as "-s" parameter to pv. Next we gzip the whole content and output the result to out.tgz file. This way "pv" knows how much data is still left to be processed and shows us that it will take yet another 4 mins 49 secs to finish.

Credit: Peteris Krumins http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

Only need to install Image Magick package.

Display a xkcd comic with its title and save it in /tmp directory

If you prefer to view the newest xkcd, use this command:

I wasn't sure how to display the image, so I thought I'd try xml for a different twist.

as unixmonkey7109 pointed out, first awk parse replaces three steps.

It's not a big line, and it *may not* work for everybody, I guess it depends on the detail of access_log configuration in your httpd.conf. I use it as a prerotate command for logrotate in httpd section so it executes before access_log rotation, everyday at midnight.

This bash function uses albumart.org to find the cover for an album. It returns an amazon.com url to the image.

Usage: albumart [artist] [album]

These arguments can be reversed and if the album name is distinct enough, it may be possible to omit the artist.

The command can be extended with wget to automatically download the matching image like this:

This function uploads images to http://omploader.org and then prints out the links to the file.

Some coloring can also be added to the command with:

Simple MAC adrress, thanks to ifconfig.

Sometimes, you don't really care about all the other information that ifconfig spits at you (however useful it may otherwise be). You just want an IP. This strips out all the crap and gives you exactly what you want.

Calculating series with awk only, no need for seq: add numbers from 1 to 100

Variations:

1+3+...+(2n-1) = n^2

1/2 + 1/4 + ... = 1

Handled all within awk. Takes the value from $PWD and constructs directory structures and runs commands against them. The gsub() call is not necessary, but added for better visibility.

If a variable DIR is given on the awk command-line, then that directory is used instead:

draw `vmstat ` data using gnuplot

This pipeline will find, sort and display all files based on mtime. This could be done with find | xargs, but the find | xargs pipeline will not produce correct results if the results of find are greater than xargs command line buffer. If the xargs buffer fills, xargs processes the find results in more than one batch which is not compatible with sorting.

Note the "-print0" on find and "-0" switch for perl. This is the equivalent of using xargs. Don't you love perl?

Note that this pipeline can be easily modified to any data produced by perl's stat operator. eg, you could sort on size, hard links, creation time, etc. Look at stat and just change the '9' to what you want. Changing the '9' to a '7' for example will sort by file size. A '3' sorts by number of links....

Use head and tail at the end of the pipeline to get oldest files or most recent. Use awk or perl -wnla for further processing. Since there is a tab between the two fields, it is very easy to process.

I have a directory containing log files. This command delete all but the 5 latest logs. Here is how it works:

* The ls -t command list all files with the latest ones at the top

* The awk's expression means: for those lines greater than 5, delete.

This will calculate a running standard deviation in one pass and should never have the possibility for overflow that can happen with other implementations. I suppose there is a potential for underflow in the corner case where the deltas are small or the values themselves are small.

This command displays a list of lines that are longer than 72 characters. I use this command to identify those lines in my scripts and cut them short the way I like it.

Removes trailing newline; colon becomes record separator and newline becomes field separator, only the first field is ever printed. Replaces empty entries with $PWD. Also prepend relative directories (like ".") with the current directory ($PWD). Can change PWD with env(1) to get tricky in (non-Bourne) scripts.

This is really fast :)

204753

real 0m0.191s

user 0m0.068s

sys 0m0.116s

Checks the Gmail ATOM feed for your account, parses it and outputs a list of unread messages.

For some reason sed gets stuck on OS X, so here's a Perl version for the Mac:

If you want to see the name of the last person, who added a message to the conversation, change the greediness of the operators like this:

Does not require input to function or complete. Number of iterations controlled by shell variable $NUM.

Have wc work on each file then add up the total with awk; get a 43% speed increase on RHEL over using "-exec cat|wc -l" and a 67% increase on my Ubuntu laptop (this is with 10MB of data in 767 files).