Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Universal configuration monitoring and system of record for IT.
Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

May 19, 2015 - A Look At The New Commandlinefu
I've put together a short writeup on what kind of newness you can expect from the next iteration of clfu. Check it out here.
March 2, 2015 - New Management
I'm Jon, I'll be maintaining and improving clfu. Thanks to David for building such a great resource!
Hide

Top Tags

Hide

Functions

Hide

Credits

Commands using uniq from sorted by
Terminal - Commands using uniq - 215 results
find /proc/*/fd -xtype f -printf "%l\n" | grep -P '^/(?!dev|proc|sys)' | sort | uniq -c | sort -n
2015-08-18 17:58:21
User: flatcap
Functions: find grep sort uniq
Tags: sort uniq find grep
1

List all open files of all processes.

.

find /proc/*/fd

Look through the /proc file descriptors

.

-xtype f

list only symlinks to file

.

-printf "%l\n"

print the symlink target

.

grep -P '^/(?!dev|proc|sys)'

ignore files from /dev /proc or /sys

.

sort | uniq -c | sort -n

count the results

.

Many processes will create and immediately delete temporary files.

These can the filtered out by adding:

... | grep -v " (deleted)$" | ...
lsof -a -d 1-99 -Fn / | grep ^n | cut -b2- | sort | uniq -c | sort -n
sudo lsof | egrep 'w.+REG' | awk '{print $10}' | sort | uniq -c | sort -n
2015-08-18 14:09:02
User: kennethjor
Functions: awk egrep sort sudo uniq
1

This command run fine on my Ubuntu machine, but on Red Hat I had to change the awk command to `awk '{print $10}'`.

phpcs --no-colors --standard=WordPress-Core -s -- $( find . -name '*.php' ) | ack -o '(?<=\()\w+(\.\w+)+(?=\)$)' | sort | uniq -c | sort -nr
groups user1 user2|cut -d: -f2|xargs -n1|sort|uniq -d
2015-03-04 19:12:27
User: swemarx
Functions: cut groups uniq xargs
3

Updated according to flatcap's suggestion, thanks!

find -not -empty -type f -printf "%-30s'\t\"%h/%f\"\n" | sort -rn -t$'\t' | uniq -w30 -D | cut -f 2 -d $'\t' | xargs md5sum | sort | uniq -w32 --all-repeated=separate
2014-10-19 02:00:55
User: fobos3
Functions: cut find md5sum sort uniq xargs
1

Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on:

find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

The new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.

ls | tr '[[:punct:][:space:]]' '\n' | grep -v "^\s*$" | sort | uniq -c | sort -bn
2014-10-14 09:52:28
User: qdrizh
Functions: grep ls sort tr uniq
Tags: sort uniq ls grep tr
3

I'm sure there's a more elegant sed version for the tr + grep section.

cat /etc/httpd/logs/access.log | awk '{ print $6}' | sed -e 's/\[//' | awk -F'/' '{print $1}' | sort | uniq -c
2014-10-13 13:39:53
User: suyashjain
Functions: awk cat sed sort uniq
1

The command will read the apache log file and fetch the virtual host requested and the number of requests.

history|awk '{print $2}'|sort|uniq -c|sort -rn|head -30|awk '!max{max=$1;}{r="";i=s=100*$1/max;while(i-->0)r=r"#";printf "%50s %5d %s %s",$2,$1,r,"\n";}'
2014-09-29 12:40:43
User: injez
Functions: awk head printf sort uniq
0

Top 30 History Command line with histogram display

tcpdump -tnn -c 2000 -i eth0 | awk -F "." '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -nr | awk ' $1 > 10 '
2014-09-26 01:15:23
User: hochmeister
Functions: awk sort tcpdump uniq
1

capture 2000 packets and print the top 10 talkers

zcat error.log.gz | sed 's^\[.*\]^^g' | sed 's^\, referer: [^\n]*^^g' | sort | uniq -c | sort -n
2014-09-24 05:26:24
User: zanhsieh
Functions: sed sort uniq zcat
Tags: sort sed uniq zcat
1

credit shall fall to this for non-gzipped version:

https://gist.github.com/marcanuy/a08d5f2d9c19ba621399

$ docker rm `(docker ps -q && docker ps -qa) | sort | uniq -u`
2014-09-16 14:08:54
User: vlf
Functions: ps rm sort uniq
0

Applies 'docker rm' to all container IDs that appear in 'docker ps -a' but not in 'docker ps' - i.e. the ones that are not running.

netstat -tn 2>/dev/null | grep :80 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr | head
awk -F: '{print $2}' access_log | sort | uniq -c
cut -f 2 -d ':' oclHashcat.pot | egrep -oi '[a-z]{1,20}' | sort | uniq > base.pot
postqueue -p | grep -A 1 "over quota" | grep @ | sort | uniq | tr --delete ' '
awk '/text to grep/{print \$1}' logs... | sort -n | uniq -c | sort -rn | head -n 100
2014-07-10 20:36:02
User: impinball
Functions: awk head sort uniq
Tags: Linux sh
0

Accepts multiple files via logs.... Substitute "text to grep" for your search string.

If you want to alias this, you could do something like this:

alias parse-logs='awk "/$1/{print \$1}" ${@[@]:1} | sort -n | uniq -c | sort -rn | head -n 100'
find . |xargs grep '<html\|<body\|<table' |sed '/~/d;s/:.*//' |sed 's/.*/mv & &.html/' |uniq >run.sh; sh run.sh
awk '/text to grep/{print $1}' "log" | sort -n | uniq -c | sort -rn | head -n 100
2014-07-09 08:48:06
User: kln0thing
Functions: awk head sort uniq
0

Original command: cat "log" | grep "text to grep" | awk '{print $1}' | sort -n | uniq -c | sort -rn | head -n 100

This is a waste of multiple cats and greps, esp when awk is being used

cat "log" | grep "text to grep" | awk '{print $1}' | sort -n | uniq -c | sort -rn | head -n 100
while true; do clear; cat /proc/[0-9]*/stat | cut -d' ' -f 3 | sort | uniq -c | awk '{print $2" "$1}'; echo '---'; sleep 1; done
export PORT=11211; ss -an4 | grep -E "ESTAB.*$PORT" | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | sort -nr
strings * |grep -v "Apple" |grep http |uniq |sed "s/<[^>]\+>//g"
history |awk '{print $3}' |awk 'BEGIN {FS="|"} {print $1}'|sort|uniq -c |sort -rn |head -10
dmesg | grep -Po 'csum failed ino\S* \d+' | sort | uniq | xargs -n 3 find / -inum 2> /dev/null
2014-03-20 06:27:15
User: Sepero
Functions: dmesg find grep sort uniq xargs
Tags: find inode btrfs
-1

Btrfs reports the inode numbers of files with failed checksums. Use `find` to lookup the file names of those inodes.