Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands using uniq from sorted by
Terminal - Commands using uniq - 210 results
netstat -an | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | awk '{ printf("%s\t%s\t",$2,$1) ; for (i = 0; i < $1; i++) {printf("*")}; print "" }'
2009-04-27 22:02:19
User: knassery
Functions: awk grep netstat sort uniq
46

Written for linux, the real example is how to produce ascii text graphs based on a numeric value (anything where uniq -c is useful is a good candidate).

emerge -epv world | grep USE | cut -d '"' -f 2 | sed 's/ /\n/g' | sed '/[(,)]/d' | sed s/'*'//g | sort | uniq > use && grep ^- use | sed s/^-// | sed ':a;N;$!ba;s/\n/ /g' > notuse && sed -i /^-/d use && sed -i ':a;N;$!ba;s/\n/ /g' use
2009-04-19 07:54:08
User: Keruspe
Functions: cut grep sed sort uniq
Tags: Gentoo emerge USE
1

This command puts all the flags of the USE variable actually used by the packages you emerged to the file "use", and those which are unused but available to the file "notuse"

grep -r "mystring" . |uniq | cut -d: -f1 | xargs sed -i "s/mystring//"
2009-04-09 12:49:01
Functions: cut grep sed uniq xargs
-3

Linux : these script enable you to edit multiple files and remove exact phrase from multiple files

history|awk '{print $2}'|awk 'BEGIN {FS="|"} {print $1}'|sort|uniq -c|sort -r
2009-04-05 13:40:56
User: kayowas
Functions: awk sort uniq
7

It will return a ranked list of your most commonly-entered commands using your command history

cat /var/log/secure | grep smtp | awk '{print $9}' | cut -f2 -d= | sort | uniq -c | sort -n | tail
2009-03-30 15:49:54
User: empulse
Functions: awk cat cut grep sort uniq
-2

Searches /var/log/secure for smtp connections then lists these by number of connections made and hosts.

cat /var/log/secure | grep sshd | grep Failed | sed 's/invalid//' | sed 's/user//' | awk '{print $11}' | sort | uniq -c | sort -n
2009-03-30 15:48:24
User: empulse
Functions: awk cat grep sed sort sshd uniq
8

Searches the /var/log/secure log file for Failed and/or invalid user log in attempts.

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
2009-03-28 21:02:26
User: tiagofischer
Functions: awk cut netstat sort uniq
14

Here is a command line to run on your server if you think your server is under attack. It prints our a list of open connections to your server and sorts them by amount.

BSD Version:

netstat -na |awk '{print $5}' |cut -d "." -f1,2,3,4 |sort |uniq -c |sort -nr
cat file.txt | sort | uniq -dc
2009-03-21 18:15:14
User: Vadi
Functions: cat sort uniq
1

Displays the duplicated lines in a file and their occuring frequency.

sort file1.txt | uniq > file2.txt
grep Mar/2009 /var/log/apache2/access.log | awk '{ print $1 }' | sort -n | uniq -c | sort -rn | head
zgrep "Failed password" /var/log/auth.log* | awk '{print $9}' | sort | uniq -c | sort -nr | less
2009-03-03 13:45:56
User: dbart
Functions: awk sort uniq zgrep
8

This command checks for the number of times when someone has tried to login to your server and failed. If there are a lot, then that user is being targeted on your system and you might want to make sure that user either has remote logins disabled, or has a strong password, or both. If your output has an "invalid" line, it is a summary of all logins from users that don't exist on your system.

gunzip -c /var/log/auth.log.*.gz | cat - /var/log/auth.log /var/log/auth.log.0 | grep "Invalid user" | awk '{print $8;}' | sort | uniq -c | less
cat file1.txt | uniq > file2.txt
find . -type f -name "*.java" -print0 | xargs -0 -n 1 svn blame | sed -n 's/^[^a-z]*\([a-z]*\).*$/\1/p' | sort | uniq -c | sort -n
sed -e "s/| /\n/g" ~/.bash_history | cut -d ' ' -f 1 | sort | uniq -c | sort -nr | head
"some line input" | sort | uniq -c | sort -nr
sed 's/[ \t]*$//' < emails.txt | tr 'A-Z' 'a-z' | sort | uniq > emails_sorted.txt
lsof | awk '{print $1}' | sort | uniq -c | sort -rn | head
netstat -alpn | grep :80 | awk '{print $4}' |awk -F: '{print $(NF-1)}' |sort | uniq -c | sort -n
last | grep -v "^$" | awk '{ print $1 }' | sort -nr | uniq -c
2009-02-18 16:38:59
User: hkyeakley
Functions: awk grep last sort uniq
15

This command takes the output of the 'last' command, removes empty lines, gets just the first field ($USERNAME), sort the $USERNAMES in reverse order and then gives a summary count of unique matches.

history | awk '{print $2}' | sort | uniq -c | sort -rn | head
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n | tail
2009-02-16 15:48:27
User: TuxOtaku
Functions: awk cut netstat sort uniq
2

This command does a tally of concurrent active connections from single IPs and prints out those IPs that have the most active concurrent connections. VERY useful in determining the source of a DoS or DDoS attack.

netstat -anl | grep :80 | awk '{print $5}' | cut -d ":" -f 1 | uniq -c | sort -n | grep -c IPHERE
2009-02-16 08:54:08
User: nullrouter
Functions: awk cut grep netstat sort uniq
3

This will tell you who has the most Apache connections by IP (replace IPHERE with the actual IP you wish to check). Or if you wish, remove | grep -c IPHERE for the full list.

netstat -pant 2> /dev/null | grep SYN_ | awk '{print $5;}' | cut -d: -f1 | sort | uniq -c | sort -n | tail -20
2009-02-16 08:49:38
3

List top 20 IP from which TCP connection is in SYN_RECV state.

Useful on web servers to detect a syn flood attack.

Replace SYN_ with ESTA to find established connections

netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | sed s/::ffff:// | cut -d: -f1 | sort | uniq -c | sort -n