Commands using uniq sorted by votes

Commands using uniq (255)

sorted by

Parse logs for IP addresses and how many hits from each IP

Accepts multiple files via logs.... Substitute "text to grep" for your search string. If you want to alias this, you could do something like this: alias parse-logs='awk "/$1/{print \$1}" ${@[@]:1} | sort -n | uniq -c | sort -rn | head -n 100'
This is sample output - yours may be different.
0

awk '/text to grep/{print \$1}' logs... | sort -n | uniq -c | sort -rn | head -n 100

impinball · 2014-07-10 20:36:02 7
Postfix: Mailboxes over quota in queue
This is sample output - yours may be different.
0

postqueue -p | grep -A 1 "over quota" | grep @ | sort | uniq | tr --delete ' '

rmontagud · 2014-07-14 08:23:43 12
Get unique basewords from Hashcat pot
This is sample output - yours may be different.
0

cut -f 2 -d ':' oclHashcat.pot | egrep -oi '[a-z]{1,20}' | sort | uniq > base.pot

Creek · 2014-07-14 21:03:22 6
Remove non-running Docker containers

Applies 'docker rm' to all container IDs that appear in 'docker ps -a' but not in 'docker ps' - i.e. the ones that are not running. Show Sample Output
This is sample output - yours may be different.
```
48eca05b7d25
6871947c3300
9f3f8123b1c6
c19b0cdd1511
e10763d0a3a2
```
0

$ docker rm `(docker ps -q && docker ps -qa) | sort | uniq -u`

vlf · 2014-09-16 14:08:54 13
List open file descriptor count by PID

Useful for when you find the system runs out of file descriptors and you want to know why. Show Sample Output
This is sample output - yours may be different.
```
  32015 java
   2567 Passenger
   1630 ruby
   1470 rtorrent
   1316 ruby
   1014 ruby
    957 python2
    689 ruby
    510 deluged
    337 postgres
```
0

lsof | cut -f 1 -d ' ' | uniq -c | sort -rn | head -n 10

bobobo1618 · 2014-11-12 09:16:19 8
Find All Email Forwarders in Zimbra Mail Server

Poor email reputation got you down? Perhaps you're unknowingly forwarding every spam email that makes it through to info@website.com to website@gmail.com. This command outputs every forwarding address set up within a Zimbra installation.
This is sample output - yours may be different.
0

for i in `zmprov -l gaa | cut -f2 -d"@" | uniq -c | awk '{print$2}'`; do zmprov -l gaa -v $i | grep -i zimbraPrefMailForwardingAddress; done

skylineservers · 2014-11-17 15:24:46 8

Show apps that use internet connection at the moment.

Show top apps that use internet, sorted by count connections and grouped by TYPE and Protocol Show Sample Output

sudo lsof -P -i -n | awk '{print $1,$5,$8}' | tail -n +2 | uniq -c | sort -nr

anton_slim · 2014-12-18 05:22:33 9

Show top of programms which are open file descriptors now, sorted by count DESC, grouped by process name and pid

Show what the process are opening many many file descriptos now, we can analyze this for the reason why server is worked slowly. Show Sample Output
This is sample output - yours may be different.
```
  10592 mysqld 3169
    301 opendkim 2604
    145 fail2ban- 3175
    128 master 3693
    113 php5-fpm 6792
    112 php5-fpm 7825
    112 php5-fpm 5872
    111 php5-fpm 2313
     94 dklab_dae 15461
     93 icecast2 2289
     78 rsyslogd 2316
     72 nginx 2599
     67 nginx 2601
     67 nginx 2600
     63 nginx 2598
     62 nginx 2602
     48 sshd 8980
     44 sshd 8976
     40 tlsmgr 27868
     39 opendkim 2603
     36 squid3 3626
     33 qmgr 3714
```
0

sudo lsof | awk '{print $1,$2}' | sort | uniq -c | sort -nr

anton_slim · 2014-12-18 05:42:15 8
Sort by size all hardlinked files in the current directory (and subdirectories)
This is sample output - yours may be different.
0

find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n

alissonf · 2015-01-27 15:03:23 8
View inodes

How to Free Inode Usage Show Sample Output
This is sample output - yours may be different.
```
     34 boot
     75 root
     94 bin
    173 sbin
    190 home
    200 tmp
    296 lib64
    891 etc
   4335 var
   6933 lib
  22942 usr
 905545 opt
```
0

sudo find . -xdev -type f | cut -d "/" -f 2 | sort | uniq -c | sort -n

alissonf · 2015-04-20 12:55:07 9
Show the number of processes by user in descending order
This is sample output - yours may be different.
0

ps -auxh | sed -r "s/\s+/ /g" | cut -d ' ' -f 1 | sort | uniq -c | sort -gr

lavidaportland · 2015-05-24 16:57:50 10
Grep for pattern & get uniq filenames

Grep for string in all files in a folder, get only the names of the file, making the file list uniq, then assingning all the file names to a variable in awk & printing on STDOUT
This is sample output - yours may be different.
0

grep -nri "pattern_to_search" folder_name/ | awk -F ":" '{print $1}' | uniq | awk 'BEGIN {f=""} {f = f" "$0} END {print f}'

ashish2 · 2015-07-19 01:00:58 11
Get the user with the top number of logins

Get an approximation of who the workstation is assigned to. You can wrap it in su - "$()" if you want to log into a shell as that user. Show Sample Output
This is sample output - yours may be different.
```
frequentuser
```
0

last | grep -i console | grep -iv 'root' | cut -f 1 -d ' ' | sort | uniq -c | sort -nr | awk '{print $2}' | head -1

phyxia · 2015-07-20 18:07:40 10
Gather list of PHPCS error messages sorted by frequency
This is sample output - yours may be different.
0

phpcs --no-colors --standard=WordPress-Core -s -- $( find . -name '*.php' ) | ack -o '(?<=$)\w+(\.\w+)+(?=$$)' | sort | uniq -c | sort -nr

westonruter · 2015-07-30 19:52:12 9
Remove duplicate files from the current directory

Remove everything after the second "rev" to only display the duplicate filenames and their hashes.
This is sample output - yours may be different.
0

sha1sum * | sort | rev | uniq -df1 | rev | cut -d" " -f3 | xargs rm

Sparkette · 2015-08-09 20:32:12 9
Print all open regular files sorted by the number of file handles open to each.
This is sample output - yours may be different.
0

lsof -a -d 1-99 -Fn / | grep ^n | cut -b2- | sort | uniq -c | sort -n

sesom42 · 2015-08-18 16:55:49 9
Download all pdf files off of a website using wget

Download all pdf files off of a website using wget. You can change the file type to download, changing the extension, as an example you can change pdf for txt in command. Show Sample Output
This is sample output - yours may be different.
```
wget http://webpage.com/p/ex1.pdf
wget http://webpage.com/p/ex2.pdf
wget http://webpage.com/p/ex3.pdf
```
0

lynx -dump -listonly http://webpage.com | awk '/pdf$/ { print "wget " $2}'| sort | uniq | sh

lopesivan · 2015-08-26 04:01:38 9
Outputs unique error messages from the apache log and count, sorted by frequency

Outputs unique error messages from the apache log, sorted by frequency. Tested on log format: [Wed Aug 26 00:00:00 2015] [error] [client xx.xx.xx.xx] Error message here Show Sample Output
This is sample output - yours may be different.
```
63 PHP PHP Warning: json_encode(): Invalid UTF-8 sequence in argument
10 Warning: Warning: preg_match() expects parameter 2 to be string, array given
7 PHP PHP Warning: Invalid argument supplied for foreach() in ...
5 PHP PHP Warning: implode(): Invalid arguments passed in ...
1 PHP PHP Parse error: syntax error, unexpected ',' in ...
```
0

cat /var/log/apache2/error.log | awk '{out=$9;for(i=9;i<=NF;i++){out=out" "$i}; print out}' | sed s/,\ referer.*// | sort | uniq -c | sort -nr

cmattoon · 2015-08-26 14:14:12 10
Shows which commands you use the most.
This is sample output - yours may be different.
0

cut -f1 -d" " ~/.bash_history | sort | uniq -c | sort -nr | head -n 30

kenorb · 2015-10-09 16:11:37 10
finds all bean ids in all xml files from the current folder

finds all xml files greps them for extracts the bean id sorts the ids creates a list of distinct entries, a unique list of bean ids Show Sample Output
This is sample output - yours may be different.
```
groupManager
groupPermissionsDelegate
groupPermissionsDelegate
handleResolver
hashRegistry
hazelcastCacheCompactor
hazelcastCacheConfigManager
```
0

find . -iname "*.xml" | xargs cat | grep '<bean[^>]\+id=' | sed 's/.*id="$[^"]\+$".*/\1/g' | sort | uniq -u

gruzilla · 2015-11-06 07:57:22 10
Count the number of unique colors there are in a websites css folder

Count the number of unique colors there are in a websites css folder (136 is way too many imho time to get people stick to a color scheme) Show Sample Output
This is sample output - yours may be different.
```
grep -r -h -o 'color: #.*' css/*|sort|uniq -c|sort -n|wc -l
136
```
0

grep -r -h -o 'color: #.*' css/*|sort|uniq -c|sort -n|wc -l

thoth · 2015-12-29 07:22:44 10
Resolve *.so dependencies in current directory and copy to /tmp/build directory

I used this to resolve runtime dependencies of an application to create self-contained tar-balls (e.g. PHP for Docker or Rkt) Requirements: - realpath - ldd - find - grep - sort - uniq - xargs
This is sample output - yours may be different.
0

for f in $(find . -name "*.so"); do ldd -v $(realpath $f) | grep -Eo "(/[a-z0-9\_.+-]+)*" | uniq | xargs -I % cp --parents % /tmp/build; done

mozzymoz · 2016-03-21 17:47:43 13
sorta apache logs by ip frequency
This is sample output - yours may be different.
0

cat access.log | awk '{print $1}' | sort -n | uniq -c | sort -nr | head -20

prees · 2016-05-05 20:52:03 13
Find 20 most frequently-used shell commands from (bash history)
This is sample output - yours may be different.
0

tr "\|\;" "\n" < ~/.bash_history | sed -e "s/^ //g" | cut -d " " -f 1 | sort | uniq -c | sort -rn | head -20

turrtle13 · 2016-07-01 19:27:12 10
4chan image batch downloader

Replace thread_link with the link of the thread you want to download images of.
This is sample output - yours may be different.
0

wget thread_link -qO - | sed 's/\ /\n/g' | grep -e png -e jpg | grep href | sed 's/href\=\"/http:/g' | sed 's/"//g' | uniq | xargs wget

hztrance · 2016-07-16 20:43:33 12
‹ First < 7 8 9 10 11 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

diff process output

Execute a process or list of commands in the given interval and output the difference in output.

Rename files in batch

write the output of a command to /var/log/user.log... each line will contain $USER, making this easy to grep for.

This command is useful if you want to copy the output of a series of commands to a file, for example if you want to pastebin the output from 'uname -a', 'lspci -vvv' and 'lsmod' for video driver trouble-shooting on your favorite Linux forum. 'log' takes all the following arguments as a command to execute, with STDOUT sent to /var/log/user.log. The command is echoed to the log before it is executed. The advantages of using logger (as opposed to appending output from commands to a file) are 1) commands are always appended to the logs... you don't have to worry about clobbering your log file accidentally by using '>' rather than '>>' 2) logs are automatically cleaned up by logrotate. The following functions allow you to mark the start and end of a section of /var/log/user.log. $ startlog() { export LOGMARK=$(date +%Y.%m.%d_%H:%M:%S); echo "$LOGMARK.START" | logger -t $USER; } then $ endlog() { echo "$LOGMARK.END" | logger -t $USER; } printlog will print all lines between $LOGMARK.START and $LOGMARK.END, removing everything that is prepended to each line by logger. $ printlog() { sudo sed -n -e "/$LOGMARK.START/,/$LOGMARK.END/p" /var/log/user.log| sed "s/.*$USER: //"; } The following command should dump just about all the information that you could possibly want about your linux configuration into the clipboard. $ startlog; for cmd in 'uname -a' 'cat /etc/issue' 'dmesg' 'lsusb' 'lspci' 'sudo lshw' 'lsmod'; do log $cmd; done; endlog; printlog | xsel --clipboard This is ready for a trip to http://pastebin.com/, and you don't have to worry about leaving temporary files lying around cluttering up $HOME. Caveats: I'm sure that startlog, endlog, and printlog could use some cleanup and error checking... there are unchecked dependencies between printlog and endlog, as well as between endlog and startlog. It might be useful for 'log' to send stderr to logger as well.

generate random password

generate password

Quick command line math

expr will give you a quick way to do basic math from the CLI. Make sure you escape things like * and leave a space between operators and digits.

trace the system calls made by a process (and its children)

strace can be invaluable in trying to figure out what the heck some misbehaving program is doing. There are number of useful flags to limit and control its output, and to attach to already running programs. (See also 'ltrace'.)

Get a Bulleted List of SVN Commits By a User for a Specifc Day (Daily Work Log)

* Replace USERNAME with the desired svn username * Replace the first YYYY-MM-DD with the date you want to get the log (this starts at the midnight event that starts this date) * Replace the second YYYY-MM-DD with the date after you want to get the log (this will end the log scan on midnight of the previous day) Example, if I want the log for December 10, 2010, I would put {2010-12-10}:{2010-12-11}

Capture FTP Credentials and Commands

Show Directories in the PATH Which does NOT Exist

I often need to know of my directory in the PATH, which one DOES NOT exist. This command answers that question * This command uses only bash's built-in commands * The parentheses spawn a new sub shell to prevent the modification of the IFS (input field separator) variable in the current shell

Get AWS temporary credentials ready to export based on a MFA virtual appliance

You might want to secure your AWS operations requiring to use a MFA token. But then to use API or tools, you need to pass credentials generated with a MFA token. This commands asks you for the MFA code and retrieves these credentials using AWS Cli. To print the exports, you can use: `awk '{ print "export AWS_ACCESS_KEY_ID=\"" $1 "\"\n" "export AWS_SECRET_ACCESS_KEY=\"" $2 "\"\n" "export AWS_SESSION_TOKEN=\"" $3 "\"" }'` You must adapt the command line to include: * $MFA_IDis ARN of the virtual MFA or serial number of the physical one * TTL for the credentials

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using uniq

Commands using uniq (255) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using uniq (255)

sorted by