Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands using sort from sorted by
Terminal - Commands using sort - 606 results
for i in emerg alert crit error warn ; do awk '$6 ~ /^\['$i'/ {print substr($0, index($0,$6)) }' error_log | sort | uniq -c | sort -n | tail -1; done
2010-04-15 21:47:18
User: zlemini
Functions: awk sort tail uniq
4

This searches the Apache error_log for each of the 5 most significant Apache error levels, if any are found the date is then cut from the output in order to sort then print the most common occurrence of each error.

rpm -qa --qf "%-30{NAME} %-10{SIZE}\n" | sort -n | less
rpm -qa --qf "%-10{SIZE} %-30{NAME}\n" | sort -nr | less
sudo awk '($9 ~ /404/)' /var/log/httpd/www.domain-access_log | awk '{print $2,$9,$7,$11}' | sort | uniq -c
2010-04-09 10:31:50
User: ninjasys
Functions: awk sort sudo uniq
Tags: log error apache
1

This command will return a full list of Error 404 pages in the given access log. The following variables have been given to awk

Hostname ($2), ERROR Code ($9), Missing Item ($7), Referrer ($11)

You can then send this into a file (>> /path/to/file), which you can open with OpenOffice as a CSV

awk '$9 == 404 {print $7}' access_log | uniq -c | sort -rn | head
2010-04-08 21:40:53
User: zlemini
Functions: awk sort uniq
8

Finds the top ten pages returning an http response code of 404 in an apache log.

cat /etc/apache2/sites-enabled/* | egrep 'ServerAlias|ServerName' | tr -s ' ' | sed 's/^\s//' | cut -d ' ' -f 2 | sed 's/www.//' | sort | uniq
2010-04-08 15:50:34
User: chronosMark
Functions: cat cut egrep sed sort tr
2

Get a list of all the unique hostnames from the apache configuration files. Handy to see what sites are running on a server. A slightly shorter version.

history | perl -F"\||<\(|;|\`|\\$\(" -alne 'foreach (@F) { print $1 if /\b((?!do)[a-z]+)\b/i }' | sort | uniq -c | sort -nr | head
2010-04-08 13:46:09
User: alperyilmaz
Functions: perl sort uniq
4

Most of the "most used commands" approaches does not consider pipes and other complexities.

This approach considers pipes, process substitution by backticks or $() and multiple commands separated by ;

Perl regular expression breaks up each line using | or < ( or ; or ` or $( and picks the first word (excluding "do" in case of for loops)

note: if you are using lots of perl one-liners, the perl commands will be counted as well in this approach, since semicolon is used as a separator

cat /etc/apache2/sites-enabled/* | egrep 'ServerAlias|ServerName' | tr -s " " | sed 's/^[ ]//g' | uniq | cut -d ' ' -f 2 | sed 's/www.//g' | sort | uniq
2010-04-08 08:51:17
User: chronosMark
Functions: cat cut egrep sed sort tr uniq
0

Get a list of all the unique hostnames from the apache configuration files. Handy to see what sites are running on a server.

find ./ -iname "*.djvu" -execdir perl -e '@s=`djvutxt \"$ARGV[0]\"\|grep -c Berlekamp`; chomp @s; print $s[0]; print " $ARGV[0]\n"' '{}' \;|sort -n
2010-04-07 11:15:26
Functions: find grep perl sort
0

Count the occurences of the word 'Berlekamp' in the DJVU files that are in the current directory, printing file names from the one having the least to the most occurences.

cut -d\ -f 1 ~/.bash_history | sort | uniq -c | sort -rn | head -n 10 | sed 's/.*/ &/g'
du -cks * | sort -rn | while read size fname; do for unit in k M G T P E Z Y; do if [ $size -lt 1024 ]; then echo -e "${size}${unit}\t${fname}"; break; fi; size=$((size/1024)); done; done
du -kd | egrep -v "/.*/" | sort -n
2010-03-30 15:40:35
User: rmbjr60
Functions: du egrep sort
-1

Thanks for the submit! My alternative produces summaries only for directories. The original post additionally lists all files in the current directory. Sometimes the files, they just clutter up the output. Once the big directory is located, *then* worry about which file(s) are consuming so much space.

TR=`free|grep Mem:|awk '{print $2}'`;ps axo rss,comm,pid|awk -v tr=$TR '{proc_list[$2]+=$1;} END {for (proc in proc_list) {proc_pct=(proc_list[proc]/tr)*100; printf("%d\t%-16s\t%0.2f%\n",proc_list[proc],proc,proc_pct);}}'|sort -n |tail -n 10
2010-03-27 01:34:50
User: d34dh0r53
Functions: awk grep sort tail
3

Prints the top 10 memory consuming processes (with children and instances aggregated) sorted by total RSS and calculates the percentage of total RAM each uses. Please note that since RSS can include shared libraries it is possible for the percentages to add up to more that the total amount of RAM, but this still gives you a pretty good idea. Also note that this does not work with the mawk version of awk, but it works fine with GNU Awk which is on most Linux systems. It also does not work on OS X.

svn stat -u | sort | sed -e "s/^M.*/\o033[31m&\o033[0m/" -e "s/^A.*/\o033[34m&\o033[0m/" -e "s/^D.*/\o033[35m&\o033[0m/"
2010-03-26 15:44:04
Functions: sed sort stat
Tags: bash svn sed
1

Use color escape sequences and sed to colorize the output of svn stat -u.

Colors: http://www.faqs.org/docs/abs/HTML/colorizing.html

svn stat characters: http://svnbook.red-bean.com/en/1.4/svn-book.html#svn.ref.svn.c.status

GNU Extensions for Escapes in Regular Expressions: http://www.gnu.org/software/sed/manual/html_node/Escapes.html

du -hs *|grep M|sort -n
2010-03-25 19:20:24
User: tuxlifan
Functions: du grep sort
3

This is easy to type if you are looking for a few (hundred) "missing" megabytes (and don't mind the occasional K slipping in)...

A variation without false positives and also finding gigabytes (but - depending on your keyboard setup - more painful to type):

du -hs *|grep -P '^(\d|,)+(M|G)'|sort -n

(NOTE: you might want to replace the ',' according to your locale!)

Don't forget that you can

modify the globbing as needed! (e.g. '.[^\.]* *' to include hidden files and directories (w/ bash))

in its core similar to:

http://www.commandlinefu.com/commands/view/706/show-sorted-list-of-files-with-sizes-more-than-1mb-in-the-current-dir

rpm --querytags | egrep -v HEADERIMMUTABLE | sort | while read tag ; do rpm -q --queryformat "$tag: [%{$tag} ]\n" -p $SomeRPMfile ; done
2010-03-25 05:40:48
Functions: egrep read rpm sort
0

If you want to relocate a package on your own, or you just want to know what those PREIN/UN and POSTIN/UN scripts will do, this will dump out all that detail simply.

You may want to expand the egrep out other verbose flags like CHANGELOGTEXT etc, as your needs require.

It isn't clear, but the formatting around $tag is important: %{$tag} just prints out the first line, while [%{$tag }] iterates thru multi-line output, joining the lines with a space (yes, there's a space between the g and } characters. To break it out for all newlines, use [%{$tag\n}] but the output will be long.

This is aside from rpm2cpio | cpio -ivd to extract the package files.

LC_ALL=C sort file | uniq -c | sort -n -k1 -r
find /dev/ -name random -exec bash -c '[ -r $0 -a -w $0 ] && dd if=$0 | sort | dd of=$0' {} \;
( du -xSk || du -kod ) | sort -nr | head
2010-03-16 04:05:14
Functions: du sort
4

No need to type out the full OR clause if you know which OS you're on, but this is easy cut-n-paste or alias to get top ten directories by singleton.

To avoid the error output from du -xSk you could always 2>/dev/null but you might miss relevant STDERR.

alias busy='my_file=$(find /usr/include -type f | sort -R | head -n 1); my_len=$(wc -l $my_file | awk "{print $1}"); let "r = $RANDOM % $my_len" 2>/dev/null; vim +$r $my_file'
2010-03-09 21:48:41
User: busybee
Functions: alias awk find head sort vim wc
22

This makes an alias for a command named 'busy'. The 'busy' command opens a random file in /usr/include to a random line with vim. Drop this in your .bash_aliases and make sure that file is initialized in your .bashrc.

git reflog show | grep '}: commit' | nl | sort -nr | nl | sort -nr | cut --fields=1,3 | sed s/commit://g | sed -e 's/HEAD*@{[0-9]*}://g'
2end () ( export LC_ALL=C; nl -n rz $1 > $1.tmp; ${EDITOR:-vi} $1.tmp; sort $1.tmp | sed -r 's/^.*[0-9]+\t+//' > $1; rm $1.tmp; )
2010-03-06 23:02:28
User: bartonski
Functions: export nl rm sed sort
0

This function is used to sort selected lines of a text file to the end of that file. Especially useful in cases where human intervention is necessary to sort out parts of a file. Let's say that you have a text file which contains the words

rough

slimy

red

fluff

dough

For whatever reason, you want to sort all words rhyming with 'tough' to the bottom of the file, and all words denoting colors to the top, while keeping the order of the rest of the file intact.

'$EDITOR' will open, showing all of the lines in the given file, numbered with '0' padding. Adding a '~' to the beginning of the line will cause the line to sort to the end of the file, adding '!' will cause it to sort to the beginning.

echo sortmeplease | perl -pe 'chomp; $_ = join "", sort split //'
ps axo rss,comm,pid | awk '{ proc_list[$2] += $1; } END { for (proc in proc_list) { printf("%d\t%s\n", proc_list[proc],proc); }}' | sort -n | tail -n 10
ps axo rss,comm,pid | awk '{ proc_list[$2]++; proc_list[$2 "," 1] += $1; } END { for (proc in proc_list) { printf("%d\t%s\n", proc_list[proc "," 1],proc); }}' | sort -n | tail -n 10
2010-03-03 16:41:05
User: d34dh0r53
Functions: awk ps sort tail
5

This command loops over all of the processes in a system and creates an associative array in awk with the process name as the key and the sum of the RSS as the value. The associative array has the effect of summing a parent process and all of it's children. It then prints the top ten processes sorted by size.