Commands using uniq sorted by votes

Commands using uniq (255)

sorted by

fork performance test

grabbed from Andrew Aylett post: http://superuser.com/questions/133313/can-i-speed-up-cygwins-fork Show Sample Output

while (true); do date --utc; done | uniq -c

hute37 · 2013-03-15 12:05:37 39

list unique file extensions recursively for a path, include extension frequency stats

Get the longest match of file extension (Ex. For 'foo.tar.gz', you get '.tar.gz' instead of '.gz') Show Sample Output
This is sample output - yours may be different.
```
100 .txt
 25 .tar.gz
 10 .vim
  3 .gitignore
```
2

find /some/path -type f -printf '%f\n' | grep -o '\..\+$' | sort | uniq -c | sort -rn

skkzsh · 2013-03-18 14:42:29 7
Lines per second in a log file
This is sample output - yours may be different.
2

tail -F some.log | perl -ne 'print time(), "\n";' | uniq -c

grana · 2013-05-11 04:51:22 6
Extract busiest times from apache access log

Analyze an Apache access log for the time period with most activity and display the hit count, requesting IP and the timestamp. May help detect a brute force dos attack.
This is sample output - yours may be different.
2

cut -d " " -f1,4 access_log | sort | uniq -c | sort -rn | head

zlemini · 2013-09-20 21:29:32 6
IP addresses connected to port 80

IP addresses and number of connections connected to port 80. Show Sample Output
This is sample output - yours may be different.
```
# netstat -tn 2>/dev/null | grep ':80 ' | awk '{print $5}' |sed -e 's/::ffff://' | cut -f1 -d: | sort | uniq -c | sort -rn | head
     67 242.127.82.189
     11 72.128.78.251
     11 92.126.132.28
      8 77.212.78.54
      4 66.269.8.217
      2 66.249.78.47
      1 84.85.188.132
      1 66.249.78.214
      1 186.249.79.17
      1 187.125.111.68
```
2

netstat -tn 2>/dev/null | grep ':80 ' | awk '{print $5}' |sed -e 's/::ffff://' | cut -f1 -d: | sort | uniq -c | sort -rn | head

copocaneta · 2014-03-12 12:43:07 5
Report the established connections for a particular port
This is sample output - yours may be different.
2

export PORT=11211; ss -an4 | grep -E "ESTAB.*$PORT" | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | sort -nr

hornetmadness · 2014-04-25 13:15:48 11
Hits per hour apache log
This is sample output - yours may be different.
2

awk -F: '{print $2}' access_log | sort | uniq -c

zlemini · 2014-07-29 15:09:27 4
Find Duplicate Files (based on size first, then MD5 hash)

Finds duplicates based on MD5 sum. Compares only files with the same size. Performance improvements on: find -not -empty -type f -printf "%s\n" | sort -rn | uniq -d | xargs -I{} -n1 find -type f -size {}c -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate The new version takes around 3 seconds where the old version took around 17 minutes. The bottle neck in the old command was the second find. It searches for the files with the specified file size. The new version keeps the file path and size from the beginning.
This is sample output - yours may be different.
2

find -not -empty -type f -printf "%-30s'\t\"%h/%f\"\n" | sort -rn -t$'\t' | uniq -w30 -D | cut -f 2 -d $'\t' | xargs md5sum | sort | uniq -w32 --all-repeated=separate

fobos3 · 2014-10-19 02:00:55 10
Read random news on the internet

Access a random news web page on the internet. The Links browser can of course be replaced by Firefox or any modern graphical web browser.
This is sample output - yours may be different.
2

links $( a=( $( lynx -dump -listonly "http://news.google.com" | grep -Eo "(http|https)://[a-zA-Z0-9./?=_-]*" | grep -v "google.com" | sort | uniq ) ) ; amax=${#a[@]} ; n=$(( `date '+%s'` % $amax )) ; echo ${a[n]} )

pascalvaucheret · 2016-07-26 11:52:12 54
Find where a kind of file is stored

In this case searches for where .desktop files are stored. The resulted is a sorted list of the top directories containing such files. Show Sample Output
This is sample output - yours may be different.
```
      1 ./usr/share/wayland-sessions/
      2 ./usr/share/gdm/greeter/autostart/
      3 ./usr/share/desktop-base/
      4 ./usr/lib/gnome-settings-daemon-3.0/gtk-modules/
      4 ./usr/share/xsessions/
      7 ./usr/share/gnome/applications/
      8 ./usr/lib/libreoffice/share/xdg/
     22 ./etc/xdg/autostart/
     45 ./usr/share/mimelnk/application/
    161 ./usr/share/applications/
```
2

find . -name '*.desktop' | sed s/[^/]*\.desktop$// | uniq -c | sort -g

T4b · 2016-09-16 12:22:38 21
See n most used commands in your bash history

You can append these commands to the bottom of the history file to access them easier with the Up key: sort ~/.bash_history|uniq -c|sort -n|tail -n 10|tr -s " "|cut -d' ' -f3- >> ~/.bash_history
This is sample output - yours may be different.
2

sort ~/.bash_history|uniq -c|sort -n|tail -n 10

knoppix5 · 2019-04-15 14:20:00 37
To get how many users logged in and logged out and how many times ?
This is sample output - yours may be different.
1

last | awk '{ print $1 }' | sort | uniq -c | grep -v wtmp

mnikhil · 2009-02-07 18:42:59 11
Display duplicated lines in a file

Displays the duplicated lines in a file and their occuring frequency.
This is sample output - yours may be different.
1

cat file.txt | sort | uniq -dc

Vadi · 2009-03-21 18:15:14 7
For Gentoo users : helping with USE / emerge

This command puts all the flags of the USE variable actually used by the packages you emerged to the file "use", and those which are unused but available to the file "notuse"
This is sample output - yours may be different.
1

emerge -epv world | grep USE | cut -d '"' -f 2 | sed 's/ /\n/g' | sed '/[(,)]/d' | sed s/'*'//g | sort | uniq > use && grep ^- use | sed s/^-// | sed ':a;N;$!ba;s/\n/ /g' > notuse && sed -i /^-/d use && sed -i ':a;N;$!ba;s/\n/ /g' use

Keruspe · 2009-04-19 07:54:08 4

Print summary of referers with X amount of occurances

This prints a summary of your referers from your logs as long as they occurred a certain number of times (in this case 500). The grep command excludes the terms, I add this in to remove results Im not interested in. Show Sample Output

awk -F\" '{print $4}' *.log | grep -v "eviljaymz\|\-" | sort | uniq -c | awk -F\ '{ if($1>500) print $1,$2;}' | sort -n

jaymzcd · 2009-05-05 22:21:04 4

Find unused IPs on a given subnet
This is sample output - yours may be different.
1

nmap -sP <subnet>.* | egrep -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' > results.txt ; for IP in {1..254} ; do echo "<subnet>.${IP}" ; done >> results.txt ; cat results.txt | sort -n -t . -k 1,1 -k 2,2 -k 3,3 -k 4,4 | uniq -u

bortoelnino · 2010-01-22 00:26:42 4
most used unix commands
This is sample output - yours may be different.
1

cut -d\ -f 1 ~/.bash_history | sort | uniq -c | sort -rn | head -n 10 | sed 's/.*/ &/g'

er0k · 2010-04-06 22:48:26 6
Display error pages in report format

This command will return a full list of Error 404 pages in the given access log. The following variables have been given to awk Hostname ($2), ERROR Code ($9), Missing Item ($7), Referrer ($11) You can then send this into a file (>> /path/to/file), which you can open with OpenOffice as a CSV
This is sample output - yours may be different.
1

sudo awk '($9 ~ /404/)' /var/log/httpd/www.domain-access_log | awk '{print $2,$9,$7,$11}' | sort | uniq -c

ninjasys · 2010-04-09 10:31:50 6
Count TCP States From Netstat

Counts TCP states from Netstat and displays in an ordered list. Show Sample Output
This is sample output - yours may be different.
```
9 ESTABLISHED
9 LISTEN
1 SYN_SENT
```
1

netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c

Kered557 · 2010-05-06 17:04:37 4
Get debian package names corresponding to latex packages used in a document
This is sample output - yours may be different.
1

grep -R usepackage * | cut -d']' -f2 | cut -s -d'{' -f 2 | sed s/"}"/.sty"}"/g | cut -d'}' -f1 | sort | uniq | xargs dpkg -S | cut -d':' -f1 | sort | uniq

prayer · 2010-05-22 19:37:26 4
To find which host made maximum number of specific tcp connections

This command is primarily going to work on linux boxes. and needs to be changed, for example IP=10\.194\.194\.2 PORT=389
This is sample output - yours may be different.
1

netstat -n | grep '^tcp.*<IP>:<PORT>' | tr " " | awk 'BEGIN{FS="( |:)"}{print $6}' | sort | uniq -c | sort -n -k1 | awk '{if ($1 >= 10){print $2}}'

guptavi · 2010-09-16 21:06:30 3
Generate list of words and their frequencies in a text file.
This is sample output - yours may be different.
1

tr A-Z a-z | tr -d "[[:punct:]][[:digit:]]" | tr ' /_' '\n' | sort | uniq -c

minduim · 2010-10-19 19:29:32 4
Download all data from Google Ngram Viewer

To learn more about Google Ngram Viewer: http://ngrams.googlelabs.com/info
This is sample output - yours may be different.
1

wget -qO - http://ngrams.googlelabs.com/datasets | grep -E href='(.+\.zip)' | sed -r "s/.*href='(.+\.zip)'.*/\1/" | uniq | while read line; do `wget $line`; done

sexyprout · 2010-12-20 17:46:04 207
Count the frequency of every word for a given file

Counts the frequency of words in a file Show Sample Output
This is sample output - yours may be different.
```
8082 the
4967 and
4061 of
3517 to
2952 a
```
1

cat YOUR_FILE|tr -d '[:punct:]'|tr '[:upper:]' '[:lower:]'|tr -s ' ' '\n'|sort|uniq -c|sort -rn

akap475a · 2011-01-14 12:34:06 3
most changed files in domains by rdiff-backup output
This is sample output - yours may be different.
1

cat /backup/hd7/rdiff-log.txt |grep Processing | awk '{ print $4 }' | sed -e 's/\// /g' | awk '{ print $1 }' |uniq -c |sort -n

kraja · 2011-03-12 18:49:09 3
‹ First < 2 3 4 5 6 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

View and review the system process tree.

The "pstree" command uses special line-drawing characters. However, when piped into the "less" pager, these are normally disabled.

Show sorted list of files with sizes more than 1MB in the current dir

Taken from here: http://linsovet.com/directory-usage-size-sorted-list

Mirror a directory structure from websites with an Apache-generated file indexes

wget/curl/friends are not good with mirroring files off websites, especially those with Apache-generated directory listings. These tools endlessly waste time downloading useless index HTML pages. lftp's mirror command does a better job without the mess.

Find the package that installed a command

Fix borked character coding in a tty.

Often you find some tty programs are messed up and confused about character encoding - 'man' is a common problem and sometimes displays weird characters for apostrophes, hyphens etc etc. Another class of programs that suffer from this are those that try to use the line drawing characters - eg RedHat's tty system admin functions such as system-config-firewall-tui system-config-network-tui etc. Adding 'LC_ALL=C' fixes most of these problems (as long as you want English! Perhaps speakers of other languages can add a comment here). For bonus points, I've added the '-c' option to the man command so that it ignores it's cache and re-computes the man page using the C locale.

which process has a port open

Pick a random line from a file

This is from perldoc -q random.*line, which says: This has a significant advantage in space over reading the whole file in. You can find a proof of this method in The Art of Computer Programming, Volume 2, Section 3.4.2, by Donald E. Knuth. Who am I to argue with Don Knuth?

make a list of movies(.m3u).

Google dictionary of word definitions

$ wget -qO - "http://www.google.com/dictionary/json?callback=dict_api.callbacks.id100&q=steering+wheel&sl=en&tl=en&restrict=pr,de&client=te" this does the actual google dictionary query, returns a JSON string encapsulated in some fancy tag $ sed 's/dict_api\.callbacks.id100.//' here we remove the tag beginning $ sed 's/,200,null)//' and here the tag end There are also some special characters which could cause problems with some JSON parsers, so if you get some errors, this is probably the case (sed is your friend). I laso like to trim the "webDefinitions" part, because it (sometimes) contains misleading information. $ sed 's/\,\"webDefinitions.*//' (but remember to append a "}" at the end, because the JSON string will be invalid) The output also contains links to mp3 files with pronounciation. As of now, this is only usable in the English language. If you choose other than English, you will only get webDefinitions (which are crap).

ionice limits process I/O, to keep it from swamping the system (Linux)

This command is somewhat similar to 'nice', but constrains I/O usage rather than CPU usage. In particular, the '-c3' flag tells the OS to only allow the process to do I/O when nothing else is pending. This dramatically increases the responsiveness of the rest of the system if the process is doing heavy I/O. There's also a '-p' flag, to set the priority of an already-running process.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using uniq

Commands using uniq (255) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using uniq (255)

sorted by