Commands tagged uniq sorted by votes

Commands tagged uniq (64)

sorted by

Read a tcpdump file and count SYN packets to port 80, Order column by destination.
This is sample output - yours may be different.
1

tcpdump -ntr NAME_OF_CAPTURED_FILE.pcap 'tcp[13] = 0x02 and dst port 80' | awk '{print $4}' | tr . ' ' | awk '{print $1"."$2"."$3"."$4}' | sort | uniq -c | awk ' {print $2 "\t" $1 }'

efuoax · 2012-08-22 21:26:10 8
most used unix commands
This is sample output - yours may be different.
1

awk '{print $1}' ~/.bash_history | sort | uniq -c | sort -rn | head -n 10

nesses · 2013-05-03 16:24:30 7
netstat with group by (ip adress)

Same as the rest, but handle IPv6 short IPs. Also, sort in the order that you're probably looking for. Show Sample Output
This is sample output - yours may be different.
```
    239 192.168.26.242
    129 172.16.12.3
     12 192.168.26.241
     12 172.16.12.2
      2 127.0.0.1
      1 172.16.12.66
      1 172.16.115.246
```
1

netstat -ntu | awk ' $5 ~ /^(::ffff:|[0-9|])/ { gsub("::ffff:","",$5); print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr

mrwulf · 2013-09-10 19:28:06 7
calculate how much bogomips one cpu core has (assuming you have 4 cores).

on some distro's you have to replace "BogoMIPS" with "bogomips". Show Sample Output
This is sample output - yours may be different.
```
#([04:33:58][root@fritz:~]#)cat /proc/cpuinfo | grep BogoMIPS | uniq | sed 's/^.*://g' | awk '{print($1 / 4) }'
52.8375
```
1

cat /proc/cpuinfo | grep BogoMIPS | uniq | sed 's/^.*://g' | awk '{print($1 / 4) }'

derpat · 2014-03-01 03:44:03 7

Parse compressed apache error log file and show top errors

credit shall fall to this for non-gzipped version: https://gist.github.com/marcanuy/a08d5f2d9c19ba621399 Show Sample Output

zanhsieh · 2014-09-24 05:26:24 8

tcpdump top 10 talkers

capture 2000 packets and print the top 10 talkers
This is sample output - yours may be different.
1

tcpdump -tnn -c 2000 -i eth0 | awk -F "." '{print $1"."$2"."$3"."$4}' | sort | uniq -c | sort -nr | awk ' $1 > 10 '

hochmeister · 2014-09-26 01:15:23 13
Print all open regular files sorted by the number of file handles open to each.

List all open files of all processes. . find /proc/*/fd Look through the /proc file descriptors . -xtype f list only symlinks to file . -printf "%l\n" print the symlink target . grep -P '^/(?!dev|proc|sys)' ignore files from /dev /proc or /sys . sort | uniq -c | sort -n count the results . Many processes will create and immediately delete temporary files. These can the filtered out by adding: ... | grep -v " (deleted)$" | ... Show Sample Output
This is sample output - yours may be different.
```
...
3 /var/log/squid/cache.log
5 /home/flatcap/.cache/tracker/meta.db-shm
5 /home/flatcap/.mozilla/firefox/bjt6fky6.default/places.sqlite
5 /home/flatcap/.mozilla/firefox/bjt6fky6.default/places.sqlite-wal
7 /home/flatcap/.cache/tracker/meta.db
7 /home/flatcap/.cache/tracker/meta.db-wal
```
1

find /proc/*/fd -xtype f -printf "%l\n" | grep -P '^/(?!dev|proc|sys)' | sort | uniq -c | sort -n

flatcap · 2015-08-18 17:58:21 11
processes per user counter

No need for sort Show Sample Output
This is sample output - yours may be different.
```
     31 somebody
      2 avahi
      1 colord
      1 libvirt-dnsmasq
      1 messagebus
      1 nobody
    181 root
      1 rtkit
      4 sddm
      1 syslog
      1 systemd-timesync
      1 usbmux
      1 whoopsie
```
1

ps hax -o user --sort user | uniq -c

sharkhands · 2016-09-15 23:31:51 15
worse alternative to <ctrl+r>

worse alternative to ctrl+r: grep the history removing duplicates without sorting (case insensitive search). Show Sample Output
This is sample output - yours may be different.
```
head ~/.bash_history # history sample for reference:
echo "duplicate"
awk --help
echo "duplicate"
echo "ctrl+r is better"

# input:
memo echo

# output:
echo "duplicate"
echo "ctrl+r is better"
```
1

function memo() { awk '! seen[$0]++' <<< $(grep -i "$@" ~/.bash_history ); }

mikhail · 2019-04-04 00:54:43 37
Find failures with journalctl

Sometimes things break. You can find the most recent errors using a combination of journalctl, along with the classic tools sort and uniq Show Sample Output
This is sample output - yours may be different.
```
898172 "/usr/bin/dockerd"
    752 "/usr/local/sbin/node_exporter"
     30 "/usr/bin/gnome-shell"
     26 "/usr/bin/cat"
     22 "/usr/libexec/gsd-media-keys"
[...]
```
1

journalctl --no-pager --since today --grep 'fail|error|fatal' --output json | jq '._EXE' | sort | uniq -c | sort --numeric --reverse --key 1

mikhail · 2021-12-22 22:27:17 268
list all file extensions in a directory

If we want files with more than one extension, like .tar.gz, only appear the latest, .gz: ls -Xp /path/to/dir | grep -Eo "\.[^./]+$" | uniq Show Sample Output
This is sample output - yours may be different.
```
.tar.gz
.py
.pyc
```
0

ls -Xp /path/to/dir | grep -Eo "\.[^/]+$" | uniq

karpoke · 2010-08-12 16:32:54 7
Generate list of words and their frequencies in a text file.

Gives the same results as the command by putnamhill using nine less characters.
This is sample output - yours may be different.
0

tr -cs A-Za-z '\n' | sort | uniq -ci

TheMightyBuzzard · 2010-10-20 04:12:58 6
netstat with group by (ip adress)

count connections, group by IP and port
This is sample output - yours may be different.
0

netstat -nt | awk -F":" '{print $2}' | sort | uniq -c

sreimann · 2011-07-05 07:53:29 4
Realtime apache hits per second
This is sample output - yours may be different.
0

tcpflow -c port 80 | grep Host

kplimack · 2012-01-05 05:38:57 6
Find duplicate UID in /etc/passwd

Detect duplicate UID in you /etc/passwd (or GID in /etc/group file). Duplicate UID is often forbidden for it can be a security breach. Show Sample Output
This is sample output - yours may be different.
```
user@host:/etc# awk -F: '{print $3}' /etc/passwd | sort |uniq -d
203
user@host:/etc# 
```
0

awk -F: '{print $3}' /etc/passwd | sort |uniq -d

ultips · 2012-01-17 11:16:35 4
See who are using a specific port

See who is using a specific port. Especially when you're using AIX. In Ubuntu, for example, this can easily be seen with the netstat command. Show Sample Output
This is sample output - yours may be different.
```
Usually a nice list of IP addresses :-)
```
0

netstat -Aan | grep .80 | grep -v 127.0.0.1 | grep EST | awk '{print $6}' | cut -d "." -f1,2,3,4 | sort | uniq

janvanderwijk · 2012-02-03 13:54:11 5
Report summary of string occurrence by time period (hour)

useful to count events in logs Show Sample Output
This is sample output - yours may be different.
```
     28 2012-02-24 08
    122 2012-02-24 09
    110 2012-02-24 10
     95 2012-02-24 11
```
0

cat z.log | grep Timeout | cut -d ':' -f1 | sort | uniq -c

hute37 · 2012-02-28 17:22:17 11
Report summary of string occurrence by time period (hour) - alternate

useful to count events in logs @see: http://www.commandlinefu.com/commands/view/10327/report-summary-of-string-occurrence-by-time-period-hour#comment Show Sample Output
This is sample output - yours may be different.
```
2012-02-24 08 28
2012-02-24 09 122
2012-02-24 10 110
2012-02-24 11 95
```
0

cat z.log | cut -d ':' -f1 | sort | uniq | xargs -l1 -iFF echo 'echo FF $(cat z.log | grep -e "^FF" | grep -e Timeout | wc -l )' | bash

hute37 · 2012-02-28 17:25:14 4

Count loglines by time (minute)

depends on date format locale ... Show Sample Output

cat z.log | cut -d ':' -f1,2 | uniq -c

hute37 · 2012-02-28 17:29:46 4

list unique file extensions recursively for a path, include extension frequency stats

the find -printf "%f\n" prints just the file name from the given path. This means directory paths which contain extensions will not be considered. Show Sample Output
This is sample output - yours may be different.
```
.m3u
.M3U
.m4a
.m4v
.mp3
.Mp3
.MP3
```
0

find /some/path -type f -and -printf "%f\n" | egrep -io '\.[^.]*$' | sort | uniq -c | sort -rn

kyle0r · 2012-04-02 19:25:35 4
Unique sort

Uniq command is mostly used in combination with sort command, as uniq removes duplicates only from a sorted file. i.e In order for uniq to work, all the duplicate entries should be in the adjacent lines. Show Sample Output
This is sample output - yours may be different.
```
Can also use -u option of sort
```
0

sort namesd.txt | uniq

ankush108 · 2012-06-26 19:22:34 7
Number of open connections per ip.

Count on a specific port (80) - FreeBSD friendly. Show Sample Output
This is sample output - yours may be different.
```
 25
```
0

netstat -an | grep 80 | wc -l

mrhassell · 2012-09-05 06:17:09 6
Find Duplicate Files (based on size first, then MD5 hash)

This is a modified version of the OP, wrapped into a bash function. This version handles newlines and other whitespace correctly, the original has problems with the thankfully rare case of newlines in the file names. It also allows checking an arbitrary number of directories against each other, which is nice when the directories that you think might have duplicates don't have a convenient common ancestor directory.
This is sample output - yours may be different.
0

find-duplicates () { find "$@" -not -empty -type f -printf "%s\0" | sort -rnz | uniq -dz | xargs -0 -I{} -n1 find "$@" -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate; }

mpeschke · 2013-01-23 23:20:26 6
list unique file extensions recursively for a path, include extension frequency stats

If you have GNU findutils, you can get only the file name with find /some/path -type f -printf '%f\n' instead of find /some/path -type f | gawk -F/ '{print $NF}' Show Sample Output
This is sample output - yours may be different.
```
100 txt
 25 gz
 10 vim
  3 gitignore
```
0

find /some/path -type f | gawk -F/ '{print $NF}' | gawk -F. '/\./{print $NF}' | sort | uniq -c | sort -rn

skkzsh · 2013-03-18 14:40:26 5
Convert uniq output to json
This is sample output - yours may be different.
0

uniq -c | sed -r 's/([0-9]+)\s(.*)/"\2": \1,/;$s/,/\n}/;1i{'

qistoph · 2013-04-13 16:48:59 8
< 1 2 3 >

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

sudo for launching gui apps in background

Need package: gksu Note: Launching gui app in background that needs sudo, won't work great with our old friendly style of launching: $ sudo gedit /etc/passwd & because this would put sudo in background ! Using gksudo as demonstrated, would popup a gui sudo window. May be this is a common knowledge, but not knowing this frustrated me during my newbie year.

Check if your desired password is already available in haveibeenpwnd database. This command uses the API provided by HIBP

List mp3 files with less than 320 kbps bitrate.

export iPad App list to txt file

This will generate the same output without changing the current directory, and filepath will be relative to the current directory. Note: it will (still) fail if your iTunes library is in a non-standard location.

Show sections of a man page.

Uses the formatting of a man page to show an outline of its headers and sub-headers.

Get MD5 checksum from a pipe stream and do not alter it

scan folder to check syntax error in php files

Print a single route to a destination and its contents exactly as the kernel sees it

Useful to determine the source_ip of outgoing packages to a certain destination

Which processes are listening on a specific port (e.g. port 80)

swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

list block devices

Shows all block devices in a tree with descruptions of what they are.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged uniq

Commands tagged uniq (64) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged uniq (64)

sorted by