Commands using uniq (255)
-
Accepts multiple files via
logs.... Substitute "text to grep" for your search string. If you want to alias this, you could do something like this:alias parse-logs='awk "/$1/{print \$1}" ${@[@]:1} | sort -n | uniq -c | sort -rn | head -n 100'0awk '/text to grep/{print \$1}' logs... | sort -n | uniq -c | sort -rn | head -n 100 -
0postqueue -p | grep -A 1 "over quota" | grep @ | sort | uniq | tr --delete ' '
-
0cut -f 2 -d ':' oclHashcat.pot | egrep -oi '[a-z]{1,20}' | sort | uniq > base.pot
-
Applies 'docker rm' to all container IDs that appear in 'docker ps -a' but not in 'docker ps' - i.e. the ones that are not running. Show Sample Output
0$ docker rm `(docker ps -q && docker ps -qa) | sort | uniq -u` -
Useful for when you find the system runs out of file descriptors and you want to know why. Show Sample Output
0lsof | cut -f 1 -d ' ' | uniq -c | sort -rn | head -n 10 -
Poor email reputation got you down? Perhaps you're unknowingly forwarding every spam email that makes it through to info@website.com to website@gmail.com. This command outputs every forwarding address set up within a Zimbra installation.
0for i in `zmprov -l gaa | cut -f2 -d"@" | uniq -c | awk '{print$2}'`; do zmprov -l gaa -v $i | grep -i zimbraPrefMailForwardingAddress; done -
Show top apps that use internet, sorted by count connections and grouped by TYPE and Protocol Show Sample Output
0sudo lsof -P -i -n | awk '{print $1,$5,$8}' | tail -n +2 | uniq -c | sort -nr -
Show what the process are opening many many file descriptos now, we can analyze this for the reason why server is worked slowly. Show Sample Output
0sudo lsof | awk '{print $1,$2}' | sort | uniq -c | sort -nr -
0find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n
-
How to Free Inode Usage Show Sample Output
0sudo find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n -
0ps -auxh | sed -r "s/\s+/ /g" | cut -d ' ' -f 1 | sort | uniq -c | sort -gr
-
Grep for string in all files in a folder, get only the names of the file, making the file list uniq, then assingning all the file names to a variable in awk & printing on STDOUT
0grep -nri "pattern_to_search" folder_name/ | awk -F ":" '{print $1}' | uniq | awk 'BEGIN {f=""} {f = f" "$0} END {print f}' -
Get an approximation of who the workstation is assigned to. You can wrap it in su - "$()" if you want to log into a shell as that user. Show Sample Output
0last | grep -i console | grep -iv 'root' | cut -f 1 -d ' ' | sort | uniq -c | sort -nr | awk '{print $2}' | head -1 -
0phpcs --no-colors --standard=WordPress-Core -s -- $( find . -name '*.php' ) | ack -o '(?<=\()\w+(\.\w+)+(?=\)$)' | sort | uniq -c | sort -nr
-
Remove everything after the second "rev" to only display the duplicate filenames and their hashes.
0sha1sum * | sort | rev | uniq -df1 | rev | cut -d" " -f3 | xargs rm -
0lsof -a -d 1-99 -Fn / | grep ^n | cut -b2- | sort | uniq -c | sort -n
-
Download all pdf files off of a website using wget. You can change the file type to download, changing the extension, as an example you can change pdf for txt in command. Show Sample Output
0lynx -dump -listonly http://webpage.com | awk '/pdf$/ { print "wget " $2}'| sort | uniq | sh -
Outputs unique error messages from the apache log, sorted by frequency. Tested on log format: [Wed Aug 26 00:00:00 2015] [error] [client xx.xx.xx.xx] Error message here Show Sample Output
0cat /var/log/apache2/error.log | awk '{out=$9;for(i=9;i<=NF;i++){out=out" "$i}; print out}' | sed s/,\ referer.*// | sort | uniq -c | sort -nr -
0cut -f1 -d" " ~/.bash_history | sort | uniq -c | sort -nr | head -n 30
-
finds all xml files greps them for extracts the bean id sorts the ids creates a list of distinct entries, a unique list of bean ids Show Sample Output
0find . -iname "*.xml" | xargs cat | grep '<bean[^>]\+id=' | sed 's/.*id="\([^"]\+\)".*/\1/g' | sort | uniq -u -
Count the number of unique colors there are in a websites css folder (136 is way too many imho time to get people stick to a color scheme) Show Sample Output
0grep -r -h -o 'color: #.*' css/*|sort|uniq -c|sort -n|wc -l -
I used this to resolve runtime dependencies of an application to create self-contained tar-balls (e.g. PHP for Docker or Rkt) Requirements: - realpath - ldd - find - grep - sort - uniq - xargs
0for f in $(find . -name "*.so"); do ldd -v $(realpath $f) | grep -Eo "(/[a-z0-9\_.+-]+)*" | uniq | xargs -I % cp --parents % /tmp/build; done -
0cat access.log | awk '{print $1}' | sort -n | uniq -c | sort -nr | head -20
-
0tr "\|\;" "\n" < ~/.bash_history | sed -e "s/^ //g" | cut -d " " -f 1 | sort | uniq -c | sort -rn | head -20
-
Replace thread_link with the link of the thread you want to download images of.
0wget thread_link -qO - | sed 's/\ /\n/g' | grep -e png -e jpg | grep href | sed 's/href\=\"/http:/g' | sed 's/"//g' | uniq | xargs wget - ‹ First < 7 8 9 10 11 >
What's this?
commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.
Check These Out
Stay in the loop…
- Follow the Tweets.
-
Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.
» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10 - Subscribe to the feeds.
-
Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):
Subscribe to the feed for:
- » all commands
- » commands with 3 up-votes (commandlinefu3)
- » commands with 10 up-votes (commandlinefu10)
- » commands using uniq