Commands tagged uniq (64)
-
awk is evil! Show Sample Output
22ps hax -o user | sort | uniq -c -
Here is a command line to run on your server if you think your server is under attack. It prints our a list of open connections to your server and sorts them by amount. BSD Version:
netstat -na |awk '{print $5}' |cut -d "." -f1,2,3,4 |sort |uniq -c |sort -nrShow Sample Output14netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n -
... plus do a sort according frequency Show Sample Output
11find . -type f | awk -F'.' '{print $NF}' | sort| uniq -c | sort -g -
This command checks for the number of times when someone has tried to login to your server and failed. If there are a lot, then that user is being targeted on your system and you might want to make sure that user either has remote logins disabled, or has a strong password, or both. If your output has an "invalid" line, it is a summary of all logins from users that don't exist on your system. Show Sample Output
9zgrep "Failed password" /var/log/auth.log* | awk '{print $9}' | sort | uniq -c | sort -nr | less -
6tr A-Z a-z | tr -cs a-z '\n' | sort | uniq -c
-
The cut should match the relevant timestamp part of the logfile, the uniq will count the number of occurrences during this time interval. Show Sample Output
5grep <something> logfile | cut -c2-18 | uniq -c -
You can use only awk
5awk -F":" '!list[$3]++{print $3}' /etc/passwd -
Change the cut range for hits per 10 sec, minute and so on... Grep can be used to filter on url or source IP. Show Sample Output
4tail -f access_log | cut -c2-21 | uniq -c -
4tail -n2000 /var/www/domains/*/*/logs/access_log | awk '{print $1}' | sort | uniq -c | sort -n | awk '{ if ($1 > 20)print $1,$2}'
-
Works on current directory, with built-in sorting. Show Sample Output
4ls -Xp | grep -Eo "\.[^/]+$" | sort | uniq -
Reads stdin, and outputs each line only once - without sorting ahead of time. This does use more memory than your system's sort utility.
4perl -ne 'print if !$a{$_}++' -
Once you get into advanced/optimized scripts, functions, or cli usage, you will use the sort command alot. The options are difficult to master/memorize however, and when you use sort commands as much as I do (some examples below), it's useful to have the help available with a simple alias. I love this alias as I never seem to remember all the options for sort, and I use sort like crazy (much better than uniq for example). # Sorts by file permissions
find . -maxdepth 1 -printf '%.5m %10M %p\n' | sort -k1 -r -g -bS 20%00761 drwxrw---x ./tmp 00755 drwxr-xr-x . 00701 drwx-----x ./askapache-m 00644 -rw-r--r-- ./.htaccess # Shows uniq history fasthistory 1000 | sed 's/^[0-9 ]*//' | sort -fubdS 50%exec bash -lxv export TERM=putty-256color Taken from my http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html Show Sample Output3alias sorth='sort --help|sed -n "/^ *-[^-]/s/^ *\(-[^ ]* -[^ ]*\) *\(.*\)/\1:\2/p"|column -ts":"' -
I'm sure there's a more elegant sed version for the tr + grep section.
3ls | tr '[[:punct:][:space:]]' '\n' | grep -v "^\s*$" | sort | uniq -c | sort -bn -
2curl -s http://tinyurl.com/create.php?url=http://<website.url>/ | sed -n 's/.*\(http:\/\/tinyurl.com\/[a-z0-9][a-z0-9]*\).*/\1/p' | uniq
-
2grep current_state= /var/log/nagios/status.dat|sort|uniq -c|sed -e "s/[\t ]*\([0-9]*\).*current_state=\([0-9]*\)/\2:\1/"|tr "\n" " "
-
list top committers (and number of their commits) of svn repository. in this example it counts revisions of current directory. Show Sample Output
2svn log -q | grep '^r[0-9]' | cut -f2 -d "|" | sort | uniq -c | sort -nr -
Get the longest match of file extension (Ex. For 'foo.tar.gz', you get '.tar.gz' instead of '.gz') Show Sample Output
2find /some/path -type f -printf '%f\n' | grep -o '\..\+$' | sort | uniq -c | sort -rn -
Displays the duplicated lines in a file and their occuring frequency.
1cat file.txt | sort | uniq -dc -
Counts TCP states from Netstat and displays in an ordered list. Show Sample Output
1netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c -
use Linux ;) Show Sample Output
1pgrep -cu ioggstream -
Just a little simplification.
1find /path/to/dir -type f | grep -o '\.[^./]*$' | sort | uniq -
If your grep doesn't have an -o option, you can use sed instead.
1find /path/to/dir -type f -name '*.*' | sed 's@.*/.*\.@.@' | sort | uniq -
A shorter version Show Sample Output
1ls | grep -Eo "\..+" | sort -u -
1sed -e 's/[;|][[:space:]]*/\n/g' .bash_history | cut --delimiter=' ' --fields=1 | sort | uniq --count | sort --numeric-sort --reverse | head --lines=20
-
shows number of mysql bin log events (which are mysql server events) per minute, useful to check stress times postmortem Show Sample Output
1mysqlbinlog <logfiles> | grep exec | grep end_log_pos | cut -d' ' -f2- | cut -d: -f-2 | uniq -c - 1 2 3 >
What's this?
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Check These Out
Stay in the loop…
- Follow the Tweets.
-
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10 - Subscribe to the feeds.
-
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
- » all commands
- » commands with 3 up-votes (commandlinefu3)
- » commands with 10 up-votes (commandlinefu10)
- » commands tagged uniq