Commands using uniq (255)
-
Top 30 History Command line with histogram display Show Sample Output
1history|awk '{print $2}'|sort|uniq -c|sort -rn|head -30|awk '!max{max=$1;}{r="";i=s=100*$1/max;while(i-->0)r=r"#";printf "%50s %5d %s %s",$2,$1,r,"\n";}' -
The command will read the apache log file and fetch the virtual host requested and the number of requests.
1cat /etc/httpd/logs/access.log | awk '{ print $6}' | sed -e 's/\[//' | awk -F'/' '{print $1}' | sort | uniq -c -
This command run fine on my Ubuntu machine, but on Red Hat I had to change the awk command to `awk '{print $10}'`.
1sudo lsof | egrep 'w.+REG' | awk '{print $10}' | sort | uniq -c | sort -n -
List all open files of all processes. .
find /proc/*/fdLook through the /proc file descriptors .-xtype flist only symlinks to file .-printf "%l\n"print the symlink target .grep -P '^/(?!dev|proc|sys)'ignore files from /dev /proc or /sys .sort | uniq -c | sort -ncount the results . Many processes will create and immediately delete temporary files. These can the filtered out by adding:... | grep -v " (deleted)$" | ...Show Sample Output1find /proc/*/fd -xtype f -printf "%l\n" | grep -P '^/(?!dev|proc|sys)' | sort | uniq -c | sort -n -
sort -R randomize the list. head -n1 takes the first.
1links `lynx -dump -listonly "http://news.google.com" | grep -Eo "(http|https)://[a-zA-Z0-9./?=_-]*" | grep -v "google.com" | sort -R | uniq | head -n1` -
No need for sort Show Sample Output
1ps hax -o user --sort user | uniq -c -
Extract a color palette from a image useful for designers. Example usage:
extract-palette myawesomeimage.jpg 4Where the first argument is the image you want to extract a palette from. The second argument is the number of colors you want. It may be the case where you want to change the search space. In that case, change the -resize argument to a bigger or smaller result. See the ImageMagick documentation for the -resize argument. Show Sample Output1extract-palette() { convert "$1" -resize 300x -dither None -colors "$2" txt: | tail -n +2 | tr -s ' ' | cut -d ' ' -f 3 | sort | uniq -c | sort -rn | tr -s ' ' | cut -d ' ' -f 3;} -
It can be used to pinpoint the path(s) where the largest number of files resides when running out of free i-nodes Show Sample Output
1find / -type f ! -regex '^/\(dev\|proc\|run\|sys\).*' | sed 's@^\(.*\)/[^/]*$@\1@' | sort | uniq -c | sort -n | tail -n 10 -
Find which directory in one filesystem that contains most inodes or files. Show Sample Output
1find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n | tail -
1find /etc -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n
-
Sometimes things break. You can find the most recent errors using a combination of journalctl, along with the classic tools sort and uniq Show Sample Output
1journalctl --no-pager --since today --grep 'fail|error|fatal' --output json | jq '._EXE' | sort | uniq -c | sort --numeric --reverse --key 1 -
0egrep -r '(render_message|multipart).*('`find app/views -name '*.erb' | grep mailer | sed -e 's/\..*//' -e 's/.*\///' | uniq | xargs | sed 's/ /|/g'`')' app/models
-
0echo $PATH|tr : '\n'|sort|uniq -d
-
0for URL in `wget -O - http://api.flickr.com/services/feeds/photos_public.gne?tags=bitch,bw 2>/dev/null | grep -E -o "http[^ ]+?jpg" | grep -v "_m" | uniq | grep -v 'buddy' `; do FILE=`echo $URL | grep -E -o "[0-9a-z_]+\.jpg"`; curl $URL > $FILE; done;
-
This command will shorten any URL the user inputs. What makes this command different is that it utilizes 5 different services and gives you 5 different outputs: is.gd, bit.ly, u.nu, geekology.co.za, and tinyurl. curl -s http://tinyurl.com/create.php?url=$1 \ | sed -n 's/.*\(http:\/\/tinyurl.com\/[a-z0-9][a-z0-9]*\).*/\1/p' \ | uniq ; curl -s http://bit.ly/?url=$1 \ | sed -n 's/.*\(shortened-url" value="http:\/\/bit.ly\/[a-zA-Z0-9][a-zA-Z0-9]*\).*/\1/p' \ | sed -n 's/.*\(http:\/\/bit.ly\/[a-zA-Z0-9][a-zA-Z0-9]*\).*/\1/p' \ | uniq ; curl -s http://geekology.co.za/shortii/create.php?u=$1 \ | sed -n 's/.*\(http:\/\/geekology.co.za\/[a-z0-9][a-z0-9]*\).*/\1/p' \ | uniq ; curl -s http://u.nu/unu-api-simple?url=$1 \ | sed -n 's/.*\(http:\/\/u.nu\/[a-z0-9][a-z0-9]*\).*/\1/p' \ | uniq ; curl -s http://is.gd/api.php?longurl=$1 \ | sed -n 's/.*\(http:\/\/is.gd\/[a-z0-9][a-z0-9]*\).*/\1/p' \ | uniq echo ""
0curl -s http://tinyurl.com/create.php?url=$1 \ | sed -n 's/.*\(http:\/\/tinyurl.com\/[a-z0-9][a-z0-9]*\).*/\1/p' \ | uniq ; curl -s http://bit.ly/?url=$1 \ | sed -n 's/.*\(shortened-url"............... -
A bit shorter and parallelized. Depending on the speed of your cpu and your disk this may run faster. Parallel is from https://savannah.nongnu.org/projects/parallel/
0find -not -empty -type f -printf "%s\n" | sort | uniq -d | parallel find -type f -size {}c | parallel md5sum | sort | uniq -w32 --all-repeated=separate -
Useful for C projects where header file names must be unique (e.g. when using autoconf/automake), or when diagnosing if the wrong header file is being used (due to dupe file names) Show Sample Output
0find . -type f |sed "s#.*/##g" |sort |uniq -c -d -
Finds all (not just adjacent) repeated lines in a file. Show Sample Output
0LC_ALL=C sort file | uniq -c | sort -n -k1 -r -
Get a list of all the unique hostnames from the apache configuration files. Handy to see what sites are running on a server.
0cat /etc/apache2/sites-enabled/* | egrep 'ServerAlias|ServerName' | tr -s " " | sed 's/^[ ]//g' | uniq | cut -d ' ' -f 2 | sed 's/www.//g' | sort | uniq -
I created this command to give me a quick overview of how many file types a directory, and all its subdirectories, contains. It works based off file extension, rather than file(1)'s magic output, because it ended up being more accurate and less confusing. Files that don't have an ext (README) are generally not important for me to want to count, but you're free to customize this fit your needs. Show Sample Output
0printf "\n%25s%10sTOTAL\n" 'FILE TYPE' ' '; for ext in $(find . -iname \*.* | egrep -o '\.[^[:space:].]+$' | egrep -v '\.svn*' | sort -f | uniq -i); do count=$(find . -iname \*$ext | wc -l); printf "%25s%10s%d\n" $ext ' ' $count; done -
0tr A-Z a-z | tr -d 0-9\[\],\*-.?\:\"\(\)#\;\<\>\@ | tr ' /_' '\n' | sort | uniq -c
-
Gives the same results as the command by putnamhill using nine less characters.
0tr -cs A-Za-z '\n' | sort | uniq -ci -
0history | awk '{print $2}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -n | tail | sort -nr
-
Your version works fine except for someone who's interested in commands 'sudo' was prefixed to i.e. in your command, use of sudo appears as number of times sudo was used. Slight variation in my command peeks into what commands sudo was used for and counts the command (ignores 'sudo')
0history | awk '{print $2,$3}' | sed s/sudo// | awk '{print $1}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -n | tail | sort -nr -
0users | xargs -n1 echo | sort | uniq -c
- ‹ First < 4 5 6 7 8 > Last ›
What's this?
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Check These Out
Stay in the loop…
- Follow the Tweets.
-
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10 - Subscribe to the feeds.
-
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
- » all commands
- » commands with 3 up-votes (commandlinefu3)
- » commands with 10 up-votes (commandlinefu10)
- » commands using uniq