Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands tagged sort from sorted by
Terminal - Commands tagged sort - 150 results
find-duplicates () { find "$@" -not -empty -type f -printf "%s\0" | sort -rnz | uniq -dz | xargs -0 -I{} -n1 find "$@" -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate; }
2013-01-23 23:20:26
User: mpeschke
Functions: find md5sum sort uniq xargs
-1

This is a modified version of the OP, wrapped into a bash function.

This version handles newlines and other whitespace correctly, the original has problems with the thankfully rare case of newlines in the file names.

It also allows checking an arbitrary number of directories against each other, which is nice when the directories that you think might have duplicates don't have a convenient common ancestor directory.

getent passwd | cut -d: -f1 | sort
du -sh /home/*|sort -rh|head -n 10
2012-09-12 11:54:06
User: toaster
Functions: du head sort
0

the -h option of du and sort (on appropriate distrib) makes output "Human" readable and still sorted by "reversed size" (sort -rh)

sudo lastb | awk '{if ($3 ~ /([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}/)a[$3] = a[$3]+1} END {for (i in a){print i " : " a[i]}}' | sort -nk 3
2012-09-11 14:51:10
User: sgowie
Functions: awk lastb sort sudo
0

The lastb command presents you with the history of failed login attempts (stored in /var/log/btmp). The reference file is read/write by root only by default. This can be quite an exhaustive list with lots of bots hammering away at your machine. Sometimes it is more important to see the scale of things, or in this case the volume of failed logins tied to each source IP.

The awk statement determines if the 3rd element is an IP address, and if so increments the running count of failed login attempts associated with it. When done it prints the IP and count.

The sort statement sorts numerically (-n) by column 3 (-k 3), so you can see the most aggressive sources of login attempts. Note that the ':' character is the 2nd column, and that the -n and -k can be combined to -nk.

Please be aware that the btmp file will contain every instance of a failed login unless explicitly rolled over. It should be safe to delete/archive this file after you've processed it.

netstat -an | grep 80 | wc -l
find . -type f -exec ls -l --full-time {} + | sort -k 6,7
2012-08-03 22:22:51
User: quadcore
Functions: find ls sort
Tags: sort find ls
2

This sorts files in multiple directories by their modification date. Note that sorting is done at the end using "sort", instead of using the "-ltr" options to "ls". This ensures correct results when sorting a large number of files, in which case "find" will call "ls" multiple times.

find . -type f -exec du -sh {} + | sort -hr | head
find . -type f -exec ls -shS {} + | head -10
2012-07-28 17:21:46
User: erichamion
Functions: find head ls
0

This requires a version of GNU find that supports the -exec {} + action, but it seems more straightforward than the versions already posted.

find . -type f -print0 | xargs -0 du -h | sort -hr | head -10
find . -type f -print0 | xargs -0 du -h | sort -hr | head
sort namesd.txt | uniq ?cd
2012-06-26 19:23:58
User: ankush108
Functions: sort uniq
0

The following displays only the entries that are duplicates.

sort namesd.txt | uniq
2012-06-26 19:22:34
User: ankush108
Functions: sort
Tags: sort uniq unique
0

Uniq command is mostly used in combination with sort command, as

uniq removes duplicates only from a sorted file. i.e In order for uniq to

work, all the duplicate entries should be in the adjacent lines.

ls -al | sort +4n
2012-06-26 19:20:05
User: ankush108
Functions: ls sort
Tags: size sort files
0

ls -al gives all files, sort +4n sorts by 5th field numerically

sort -t: -k 2 names.txt
2012-06-26 19:15:30
User: ankush108
Functions: sort
Tags: sort sorting
0

Sort using kth column using : delimiter

netstat -tn | grep :80 | awk '{print $5}'| grep -v ':80' | cut -f1 -d: |cut -f1,2,3 -d. | sort | uniq -c| sort -n
2012-06-26 08:29:37
User: krishnan
Functions: awk cut grep netstat sort uniq
0

cut -f1,2 - IP range 16

cut -f1,2,3 - IP range 24

cut -f1,2,3,4 - IP range 24

find . -type f -exec stat -f '%m %N' {} \; | sort -n
cat /var/log/nginx/access.log | grep -oe '^[0-9.]\+' | perl -ne 'system("geoiplookup $_")' | grep -v found | grep -oe ', [A-Za-z ]\+$' | sort | uniq -c | sort -n
2012-05-08 13:28:25
User: theist
Functions: cat grep perl sort uniq
Tags: sort uniq geoip
-1

Per country GET report, based on access log. Easy to transform to unique IP

find . -type f -print0 | xargs -0 du -h | sort -hr | head -20
2012-03-30 10:21:12
User: flatcap
Functions: du find head sort xargs
7

Search for files and list the 20 largest.

find . -type f

gives us a list of file, recursively, starting from here (.)

-print0 | xargs -0 du -h

separate the names of files with NULL characters, so we're not confused by spaces

then xargs run the du command to find their size (in human-readable form -- 64M not 64123456)

| sort -hr

use sort to arrange the list in size order. sort -h knows that 1M is bigger than 9K

| head -20

finally only select the top twenty out of the list

for w in $(tr 'A-Z ,."()?!;:' 'a-z\n' < sample.txt); do echo ${#w} $w; done | sort -u | sort -n
2012-03-15 14:14:11
User: flatcap
Functions: echo sort tr
Tags: bash sort tr
0

Take a file and ,."()?!;: give a list of all the words in order of increasing length.

First of all use tr to map all alphabetic characters to lower case and also strip out any puntuation.

A-Z become a-z

,."()?!;: all become \n (newline)

I've ignored - (hyphen) and ' (apostrophe) because they occur in words.

Next use bash to print the length ${#w} and the word

Finally sort the list numerically (sort -n) and remove any duplicates (sort -u).

Note: sort -nu performs strangely on this list. It outputs one word per length.

du -s $(ls -l | grep '^d' | awk '{print $9}') | sort -nr
alias sortfast='sort -S$(($(sed '\''/MemF/!d;s/[^0-9]*//g'\'' /proc/meminfo)/2048)) $([ `nproc` -gt 1 ]&&echo -n --parallel=`nproc`)'
3

sort is way slow by default. This tells sort to use a buffer equal to half of the available free memory. It also will use multiple process for the sort equal to the number of cpus on your machine (if greater than 1). For me, it is magnitudes faster.

If you put this in your bash_profile or startup file, it will be set correctly when bash is started.

sort -S1 --parallel=2 <(echo) &>/dev/null && alias sortfast='sort -S$(($(sed '\''/MemF/!d;s/[^0-9]*//g'\'' /proc/meminfo)/2048)) $([ `nproc` -gt 1 ]&&echo -n --parallel=`nproc`)'

Alternative

echo|sort -S10M --parallel=2 &>/dev/null && alias sortfast="command sort -S$(($(sed '/MemT/!d;s/[^0-9]*//g' /proc/meminfo)/1024-200)) --parallel=$(($(command grep -c ^proc /proc/cpuinfo)*2))"
sed -e 's/[;|][[:space:]]*/\n/g' .bash_history | cut --delimiter=' ' --fields=1 | sort | uniq --count | sort --numeric-sort --reverse | head --lines=20
netstat -Aan | grep .80 | grep -v 127.0.0.1 | grep EST | awk '{print $6}' | cut -d "." -f1,2,3,4 | sort | uniq
2012-02-03 13:54:11
Functions: awk cut grep netstat sort
0

See who is using a specific port. Especially when you're using AIX. In Ubuntu, for example, this can easily be seen with the netstat command.

find . -type f|perl -lne '@x=sort {$b->[0]<=>$a->[0]}[(stat($_))[7],$_],@x;splice(@x,11);print "@{$x[0]}";END{for(@x){print "@$_"}'
2012-01-08 14:43:43
User: bazzargh
Functions: find perl
Tags: sort perl find
-2

A different approach to the problem - maintain a small sorted list, print the largest as we go, then the top 10 at the end. I often find that the find and sort take a long time, and the large file might appear near the start of the find. By printing as we go, I get better feedback. The sort used in this will be much slower on perls older than 5.8.

ps -ef | grep [j]ava | awk -F ' ' ' { print $1," ",$2,"\t",$(NF-2),"\t",$(NF-1),"\t",$NF } ' | sort -k4
2012-01-05 16:05:48
User: drockney
Functions: awk grep ps sort
Tags: sort awk grep ps
0

Tested in bash on AIX & Linux, used for WAS versions 6.0 & up. Sorts by node name.

Useful when you have vertically-stacked instances of WAS/Portal. Cuts out all the classpath/optional parameter clutter that makes a simple "ps -ef | grep java" so difficult to sort through.