Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands using awk from sorted by
Terminal - Commands using awk - 1,116 results
cat /var/log/secure | grep smtp | awk '{print $9}' | cut -f2 -d= | sort | uniq -c | sort -n | tail
2009-03-30 15:49:54
User: empulse
Functions: awk cat cut grep sort uniq
-2

Searches /var/log/secure for smtp connections then lists these by number of connections made and hosts.

cat /var/log/secure | grep sshd | grep Failed | sed 's/invalid//' | sed 's/user//' | awk '{print $11}' | sort | uniq -c | sort -n
2009-03-30 15:48:24
User: empulse
Functions: awk cat grep sed sort sshd uniq
8

Searches the /var/log/secure log file for Failed and/or invalid user log in attempts.

netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
2009-03-28 21:02:26
User: tiagofischer
Functions: awk cut netstat sort uniq
14

Here is a command line to run on your server if you think your server is under attack. It prints our a list of open connections to your server and sorts them by amount.

BSD Version:

netstat -na |awk '{print $5}' |cut -d "." -f1,2,3,4 |sort |uniq -c |sort -nr
awk '/start_pattern/,/stop_pattern/' file.txt
2009-03-28 14:28:59
User: atoponce
Functions: awk
80

I find this terribly useful for grepping through a file, looking for just a block of text. There's "grep -A # pattern file.txt" to see a specific number of lines following your pattern, but what if you want to see the whole block? Say, the output of "dmidecode" (as root):

dmidecode | awk '/Battery/,/^$/'

Will show me everything following the battery block up to the next block of text. Again, I find this extremely useful when I want to see whole blocks of text based on a pattern, and I don't care to see the rest of the data in output. This could be used against the '/etc/securetty/user' file on Unix to find the block of a specific user. It could be used against VirtualHosts or Directories on Apache to find specific definitions. The scenarios go on for any text formatted in a block fashion. Very handy.

netstat -atn | awk ' /tcp/ {printf("%s\n",substr($4,index($4,":")+1,length($4) )) }' | sed -e "s/://g" | sort -rnu | awk '{array [$1] = $1} END {i=32768; again=1; while (again == 1) {if (array[i] == i) {i=i+1} else {print i; again=0}}}'
2009-03-27 20:38:43
User: mpb
Functions: awk netstat sed sort
4

Some commands (such as netcat) have a port option but how can you know which ports are unused?

while true; do X=$Y; sleep 1; Y=$(ifconfig eth0|grep RX\ bytes|awk '{ print $2 }'|cut -d : -f 2); echo "$(( Y-X )) bps"; done
arp -s $(route -n | awk '/^0.0.0.0/ {print $2}') \ $(arp -n | grep `route -n | awk '/^0.0.0.0/ {print $2}'`| awk '{print $3}')
awk '{sum+=$1; sumsq+=$1*$1} END {print sqrt(sumsq/NR - (sum/NR)**2)}' file.dat
seq 0 0.1 20 | awk '{print $1, cos(0.5*$1)*sin(5*$1)}' | graph -T X
2009-03-24 21:46:59
User: kaan
Functions: awk seq
2

The arguments of "seq" indicate the starting value, step size, and the end value of the x-range. "awk" outputs (x, f(x)) pairs and pipes them to "graph", which is part of the "plotutils" package.

seq 6 | awk '{for(x=1; x<=5; x++) {printf ("%f ", rand())}; printf ("\n")}'
2009-03-24 21:33:38
User: kaan
Functions: awk printf seq
Tags: awk seq
3

Displays six rows and five columns of random numbers between 0 and 1. If you need only one column, you can dispense with the "for" loop.

awk '{sum1+=$1; sum2+=$2} END {print sum1/NR, sum2/NR}' file.dat
2009-03-24 21:22:14
User: kaan
Functions: awk
Tags: awk
2

This example calculates the averages of column one and column two of "file.dat". It can be easily modified if other columns are to be averaged.

seq 50| awk 'BEGIN {a=1; b=1} {print a; c=a+b; a=b; b=c}'
2009-03-24 20:39:24
User: kaan
Functions: awk seq
Tags: awk seq
13

Another combination of seq and awk. Not very efficient, but sufficiently quick.

seq 100 | awk '{sum+=$1} END {print sum}'
2009-03-24 20:30:40
User: kaan
Functions: awk seq
Tags: awk seq
4

"seq 100" outputs 1,2,..,100, separated by newlines. awk adds them up and displays the sum.

"seq 1 2 11" outputs 1,3,..,11.

Variations:

1+3+...+(2n-1) = n^2

seq 1 2 19 | awk '{sum+=$1} END {print sum}' # displays 100

1/2 + 1/4 + ... = 1

seq 10 | awk '{sum+=1/(2**$1)} END {print sum}' # displays 0.999023
ps ax | grep <processname> | grep -v grep | awk '{print $1}' | sudo xargs kill -9
/usr/sbin/arp -i eth0 | awk '{print $3}' | sed 1d
ps aux | grep 'httpd ' | awk {'print $2'} | xargs kill -9
mysql --database=dbname -B -N -e "SHOW TABLES" | awk '{print "ALTER TABLE", $1, "CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci;"}' | mysql --database=dbname &
2009-03-21 18:45:15
User: root
Functions: awk
Tags: mysql
18

This loops through all tables and changes their collations to UTF8. You should backup beforehand though in case some data is lost in the process.

sudo zcat /var/log/auth.log.*.gz | awk '/Failed password/&&!/for invalid user/{a[$9]++}/Failed password for invalid user/{a["*" $11]++}END{for (i in a) printf "%6s\t%s\n", a[i], i|"sort -n"}'
2009-03-21 06:41:59
Functions: awk printf sudo zcat
22

Show the number of failed tries of login per account. If the user does not exist it is marked with *.

ropened='p4 opened | awk -F# "{print \$1}" | p4 -x - revert'
alias opened='p4 opened | awk -F# "{print \$1}"'
2009-03-20 11:06:41
User: Alexander
Functions: alias awk
Tags: p4 SCM Perforce
0

Just type 'opened' and get all files currently opened for edit.

vos listvldb | agrep LOCKED -d RWrite | grep RWrite: | awk -F: '{print $2}' | awk '{printf("%s ",$1)} END {printf("\n")}'
2009-03-17 19:55:39
User: mpb
Functions: awk grep
0

This command shows if there are any locked AFS volumes.

The output is a list of AFS volume IDs (or nothing if there are none locked).

cat count.txt | awk '{ sum+=$1} END {print sum}'
2009-03-16 00:22:13
User: duxklr
Functions: awk cat
Tags: awk
16

Takes a input file (count.txt) that looks like:

1

2

3

4

5

It will add/sum the first column of numbers.

ls -1 | grep " " | awk '{printf("mv \"%s\" ",$0); gsub(/ /,"_",$0); printf("%s\n",$0)}' | sh # rename filenames: spaces to "_"
2009-03-15 18:42:43
User: mpb
Functions: awk grep ls rename sh
2

This command converts filenames with embedded spaces in the current directory replacing spaces with the underscore ("_") character.

ls -l|awk '{print $6,$8}'|sort -d
2009-03-13 19:00:18
User: archlich
Functions: awk ls sort
-4

Can pipe to tail or change the awk for for file size, groups, users, etc.

lsof|grep /somemount/| awk '{print $2}'|xargs kill
2009-03-12 18:42:19
User: archlich
Functions: awk grep xargs
4

This command will kill all processes using a directory. It's quick and dirty. One may also use a -9 with kill in case regular kill doesn't work. This is useful if one needs to umount a directory.