Commands using uniq sorted by votes

Commands using uniq (255)

sorted by

Retrieve a download count for URLs in apache logs
This is sample output - yours may be different.
0

zgrep 'pattern' /var/logs/apache2/access.log* | awk '{print $7}' | sort -n | uniq -c | sort -rn

funkjedi · 2016-11-30 20:14:28 15
Find the top most 5 duplicate files in a folder

To allow recursivity : find -type f -exec md5sum '{}' ';' | sort | uniq -c -w 33 | sort -gr | head -n 5 | cut -c1-7,41- Display only filenames : find -maxdepth 1 -type f -exec md5sum '{}' ';' | sort | uniq -c -w 33 | sort -gr | head -n 5 | cut -c43- Show Sample Output
This is sample output - yours may be different.
```
     50 ./file17.txt
     32 ./file44.txt
     11 ./file72.txt
      5 ./file29.txt
      3 ./file53.txt
```
0

find -maxdepth 1 -type f -exec md5sum '{}' ';' | sort | uniq -c -w 33 | sort -gr | head -n 5 | cut -c1-7,41-

MaDCOw · 2017-02-09 11:36:31 18
get all the IP addresses currently blocked by UFW (uncomplicated firewall)

Useful in scripts to compare known malicious IP addresses with what you are actually blocking. Show Sample Output
This is sample output - yours may be different.
```
217.118.95.89
217.12.113.114
```
0

ufw status | sort | uniq | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}"

harvix · 2017-02-17 12:12:44 18
Check the connection of the maximum number of IP

Check the connection of the maximum number of IP Show Sample Output
This is sample output - yours may be different.
```
      2 10.177.137.104
      2 10.139.17.154
      1 10.219.144.207
      1 10.137.2.204
      1 10.137.138.17
      1 10.130.0.238
```
0

netstat -na | grep ESTABLISHED | awk '{print$5}' | awk -F : '{print$1}' | sort | uniq -c | sort -r

casechen · 2017-03-26 12:12:09 17

Find Duplicate Files (based on size, name, and md5sum)

It works extremely fast, because it calculates md5sum only on the files that have the same size and name. But there is nothing for free - it won't find duplicates with the different names. Show Sample Output

find -type f -printf '%20s\t%100f\t%p\n' | sort -n | uniq -Dw121 | awk -F'\t' '{print $3}' | xargs -d '\n' md5sum | uniq -Dw32 | cut -b 35- | xargs -d '\n' ls -lU

ant7 · 2017-05-21 02:26:16 16

List all IP's connected to port 80
This is sample output - yours may be different.
0

netstat -tn 2>/dev/null | grep :80 | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr | head

kumarrav90 · 2017-05-28 10:57:14 15
Find every directory with too many files and subdirectory.

if you have less free inode space alert in your monitoring app you can use this command to find the any directory with too many files and subdirectory . Show Sample Output
This is sample output - yours may be different.
```
  12496 /var/www/site/wp-content/cache/autoptimize/js
  15146 /var/www/site/wp-content/uploads/2016/06
  15764 /var/www/site/wp-content/uploads/2016/07
  18068 /var/www/site/wp-content/uploads/2016/11
  2076387 /var/lib/hhvm/sessions
```
0

find / -xdev -printf '%h\n' | sort | uniq -c | sort -k 1 -n

ara4sh · 2017-09-06 05:02:28 18
Create unique IP and count list of hits against URL in Pound LB logs, redirect to file
This is sample output - yours may be different.
0

cat *pound* | grep someURL.com | awk '{print $7}'| uniq -c | sort -bgr >> uniqueIPwC.txt

MannWare · 2017-11-16 20:18:55 18
find hard linked files (duplicate inodes) recursively

If you have two sets of files that may share hard-linked files, it can be useful to identify which ones are hard links to same underlying inode (file). This command shows you all of those, sorted by inode#. In my example the two directory trees to compare share a common parent, so I run the command in that parent and just use find . to start from the current directory. If yours are in different locations, you can pass both paths to find: find /directory1 /directory2 -type f -printf '%10i %p\n' | sort | uniq -w 11 -d -D | less Show Sample Output
This is sample output - yours may be different.
```
 190186228 ./10/alpha1/2612_fsm
 190186228 ./12/alpha1/base/19108/2612_fsm
 190203401 ./10/alpha1/pg_internal.init
 190203401 ./12/alpha1/base/19108/pg_internal.init
 190206322 ./10/alpha1/3079_fsm
 190206322 ./12/alpha1/base/19108/3079_fsm
```
0

find . -type f -printf '%10i %p\n' | sort | uniq -w 11 -d -D | less

dmmst19 · 2020-09-13 22:12:58 385
Generate CHECK TABLE statements for all MySQL database tables on a server

This command will generate "CHECK TABLE `db_name.table_name` ;" statements for all tables present in databases on a MySQL server, which can be piped into the mysql command. (Can also be altered to perform OPTIMIZE and REPAIR functions.) Tested on MySQL 4.x and 5.x systems in a Linux environment under bash. Show Sample Output
This is sample output - yours may be different.
```
CHECK TABLE `mysql.columns_priv` ;
CHECK TABLE `mysql.db` ;
CHECK TABLE `mysql.func` ;
CHECK TABLE `mysql.help_category` ;
CHECK TABLE `mysql.help_keyword` ;
CHECK TABLE `mysql.help_relation` ;
CHECK TABLE `mysql.help_topic` ;
CHECK TABLE `mysql.host` ;
CHECK TABLE `mysql.tables_priv` ;
CHECK TABLE `mysql.time_zone` ;
CHECK TABLE `mysql.time_zone_leap_second` ;
CHECK TABLE `mysql.time_zone_name` ;
CHECK TABLE `mysql.time_zone_transition` ;
CHECK TABLE `mysql.time_zone_transition_type` ;
CHECK TABLE `mysql.user` ;
CHECK TABLE `mysql.user_info` ;
```
-1

DD=`cat /etc/my.cnf | sed "s/#.*//g;" | grep datadir | tr '=' ' ' | gawk '{print $2;}'` && ( cd $DD ; find . -mindepth 2 | grep -v db\.opt | sed 's/\.\///g; s/\....$//g; s/\//./;' | sort | uniq | tr '/' '.' | gawk '{print "CHECK TABLE","`"$1"`",";";}' )

atcroft · 2009-07-25 03:42:31 15
Show apps that use internet connection at the moment.

Can be used to discover what programms create internet traffic. Skip the part after awk to get more details. Has anyone an idea why the uniq doesn't work propperly here (see sample output)? Show Sample Output
This is sample output - yours may be different.
```
java          
pidgin         
pidgin         
pidgin         
pidgin         
pidgin         
thunderbird-b 
thunderbird-b 
thunderbird-b 
thunderbird-b 
```
-1

netstat -lantp | grep -i establ | awk -F/ '{print $2}' | uniq | sort

ktoso · 2009-09-19 13:54:36 7
Prints per-line contribution per author for a GIT repository

You'll run into trouble if you have files w/ missing newlines at the end. I tried to use PAGER='sed \$q' git blame and even PAGER='sed \$q' git -p blame to force a newline at the end, but as soon as the output is redirected, git seems to ignore the pager.
This is sample output - yours may be different.
-1

git ls-files | while read i; do git blame $i | sed -e 's/^[^(]*(//' -e 's/^$[^[:digit:]]*$[[:space:]]\+[[:digit:]].*/\1/'; done | sort | uniq -ic | sort -nr

pipping · 2009-10-25 09:40:01 4
intersection between two files

print members both in file1 and file2
This is sample output - yours may be different.
-1

sort file1 file2 | uniq -d

emacs · 2010-05-28 10:25:31 6
Find most used focal lengths in a directory of photos

First column is number of photos, second column is the focal length. Show Sample Output
This is sample output - yours may be different.
```
     17 17.0
      2 47.0
      1 50.0
      4 53.0
      6 78.0
      3 85.0
```
-1

exiv2 *JPG | grep Focal | awk '{print $5}' | sort -n | uniq -c

ivanatora · 2011-07-05 11:25:29 8
Per country GET report, based on access log. Easy to transform to unique IP

Per country GET report, based on access log. Easy to transform to unique IP Show Sample Output
This is sample output - yours may be different.
```
     71 , Poland
    116 , France
    300 , United States
   2426 , Spain
```
-1

cat /var/log/nginx/access.log | grep -oe '^[0-9.]\+' | perl -ne 'system("geoiplookup $_")' | grep -v found | grep -oe ', [A-Za-z ]\+$' | sort | uniq -c | sort -n

theist · 2012-05-08 13:28:25 3
Tshark to Generate Top Talkers by #TCP conv started per second.

#_connects src_IP dst_IP When_It_Happened_Secs Show Sample Output
This is sample output - yours may be different.
```
 23 208.127.211.141 170.135.216.181 7
  22 67.142.174.23 170.135.216.181 12
  20 157.93.242.50 170.135.216.181 6
  19 67.142.174.23 170.135.216.181 11
  17 67.160.80.189 170.135.216.181 11
  
```
-1

tshark -qr [cap] -z conv,tcp | awk '{printf("%s:%s:%s\n",$1,$3,$10)}' | awk -F: '{printf("%s %s %s\n",$1,$3,substr($5,1,length($5)-10))}' | sort | uniq -c | sort -nr

knassery · 2012-10-16 06:39:01 4
Show number of connections per remote IP

Output contains also garbage (text parts from netstat's output) but it's good enough for quick check who's overloading your server.
This is sample output - yours may be different.
-1

netstat -antu | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | sort -n

wejn · 2013-04-08 19:46:41 9
check your up to date delicious links.

This commands queries the delicious api then runs the xml through xml2, grabs the urls cuts out the first two columns, passes through uniq to remove duplicates if any, and then goes into linkchecker who checks the links. the links go the blacklist in ~/.linkchecker/blacklist. please see the manual pages for further info peeps. I took me a few days to figure this one out. I how you enjoy it. Also don't run these api more then once a few seconds you can get banned by delicious see their site for info. ~updated for no recursive Show Sample Output
This is sample output - yours may be different.
```
1 http://www.jokosher.org/
1 http://www.learnopenoffice.org/calccontents.htm
1 http://ubcd4win.com/forum/index.php?showtopic=9630
1 http://212.182.0.171/dwww/
1 http://www.smarthomeforum.com/start/default.asp
1 http://www.sjrcc.edu/
1 http://curvedns.on2it.net/docs
1 http://www.smarthomeforum.com/
```
-1

curl -k https://Username:Password@api.del.icio.us/v1/posts/all?red=api | xml2| \grep '@href' | cut -d\= -f 2- | sort | uniq | linkchecker -r0 --stdin --complete -v -t 50 -F blacklist

bbelt16ag · 2013-05-04 17:43:21 8
Btrfs: Find file names with checksum errors

Btrfs reports the inode numbers of files with failed checksums. Use `find` to lookup the file names of those inodes.
This is sample output - yours may be different.
-1

dmesg | grep -Po 'csum failed ino\S* \d+' | sort | uniq | xargs -n 3 find / -inum 2> /dev/null

Sepero · 2014-03-20 06:27:15 10
Parse logs for IP addresses and how many hits from each IP
This is sample output - yours may be different.
-1

cat "log" | grep "text to grep" | awk '{print $1}' | sort -n | uniq -c | sort -rn | head -n 100

jmorganwalker · 2014-07-08 15:07:24 10
display IP addresses with 5 or more unsuccessful login attempts today

display IP's that unsuccessfully attempted to login 5 or more times today may want to filter any trusted IP's and the localhost useful for obtaining a list IP addresses to block on the firewall Show Sample Output
This is sample output - yours may be different.
```
the output is zero or more IP addresses
```
-1

lastb -i | grep "$(date '+%a %b %d')" | awk '{ print $3 }' | sort | uniq -c | awk '{ if ($1 >= 5) print $2; }'

forestb · 2015-11-20 07:19:20 10
Checking total connections to each Ip inserver

Useful to check DDoS attacks on servers. Show Sample Output
This is sample output - yours may be different.
```
1 127.0.0.1
 8 192.168.1.147
 21 192.168.1.146
```
-2

netstat -alpn | grep :80 | awk '{print $4}' |awk -F: '{print $(NF-1)}' |sort | uniq -c | sort -n

nitins · 2009-02-19 04:59:32 9
List SMTP connections by host

Searches /var/log/secure for smtp connections then lists these by number of connections made and hosts.
This is sample output - yours may be different.
-2

cat /var/log/secure | grep smtp | awk '{print $9}' | cut -f2 -d= | sort | uniq -c | sort -n | tail

empulse · 2009-03-30 15:49:54 7
Rank top 10 most frequently used commands

I make an extensive use of sudo, so I had to exclude the sudo part of the command history
This is sample output - yours may be different.
-2

history | sed -e 's/^sudo //' | awk '{print $2}' | sort | uniq -c | sort -rn | head

bibe · 2012-01-07 22:29:54 3
Analyse a PHP file for instantations and static calls

This grabs all lines that make an instantation or static call, then filters out the cruft and displays a summary of each class called and the frequency. Show Sample Output
This is sample output - yours may be different.
```
1 parent
1 product_Item
1 promotion_Base
1 swc_Response
2 product_ViewHelper
2 product_ViewTransformer
4 promotion_Tab
5 site_Template
6 self
```
-3

grep -o "$new \(\w\+$\|\w\+::\)" file.php | sed 's/new \|:://' | sort | uniq -c | sort

root · 2009-01-26 12:08:47 57
‹ First < 8 9 10 11 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Close a hanging ssh session

~ is the ssh control character. If you type it after a newline, you can example force close the ssh session. , ~, then . If you want more info, type ~? when you are connecting using ssh. Very useful to kill of any hanging ssh sessions when nothing else works.

STAT Function showing ALL info, stat options, and descriptions

This shows every bit of information that stat can get for any file, dir, fifo, etc. It's great because it also shows the format and explains it for each format option. If you just want stat help, create this handy alias 'stath' to display all format options with explanations. $ alias stath="stat --h|sed '/Th/,/NO/!d;/%/!d'" To display on 2 lines: $ ( F=/etc/screenrc N=c IFS=$'\n'; for L in $(sed 's/%Z./%Z\n/'

Add all unversioned files to svn

No need for grep, let awk do the match. This will not behave properly if the filenames contains whitespace, which is awk's default field separator.

Always tail/edit/grep the latest file in a directory of timestamped files

zsh only If you have this command in your history, you can always re-run it and have it reference the latest file. The glob matches all timestamped files and then the resulting array is sorted by modification time (m) and then the first element in the sorted array is chosen (the latest)

Sort IPV4 ip addresses

Find all clients connected to HTTP or HTTPS ports

It finds, specifically, the connections to the HTTP and HTTPS ports as source ports. You can check for destination ports as well.

Calculate days on which Friday the 13th occurs

I removed the dependency of the English language

Create user add lines from partial passwd file

Handy if you are installing a new server or recovering an old one and you have a passwd file with the accounts you want to add to the server. If you edit the file so that only the accounts that you want to add are left this line will spit out the correct useradd lines. The uid, gecos and shell will be preserved.

Install pip with Proxy

Installs pip packages defining a proxy

Display a markdown file in a new tab in firefox

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using uniq

Commands using uniq (255) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using uniq (255)

sorted by