Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Commands using awk from sorted by
Terminal - Commands using awk - 1,163 results
awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' | sed '/^$/d' > file.html
2010-04-20 13:27:47
User: tamouse
Functions: awk sed
Tags: Linux awk html
1

previous version leaves lots of blank lines

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' file.html
2010-04-20 10:54:03
User: sata
Functions: awk
Tags: Linux awk html
4

Case Insensitive! and Works even if the "<title>...</title>" spans over multiple line.

Simple! :-)

find ~/.mozilla/firefox/*/Cache -exec file {} \; | awk -F ': ' 'tolower($2)~/mpeg/{print $1}'
2010-04-19 06:59:55
User: sata
Functions: awk file find
2

Grab a list of MP3s (with full path) out of Firefox's cache

Ever gone to a site that has an MP3 embedded into a pesky flash player, but no download link? Well, this one-liner will yank the *full path* of those tunes straight out of FF's cache in a clean list.

Shorter and Intuitive version of the command submitted by (TuxOtaku)

for i in emerg alert crit error warn ; do awk '$6 ~ /^\['$i'/ {print substr($0, index($0,$6)) }' error_log | sort | uniq -c | sort -n | tail -1; done
2010-04-15 21:47:18
User: zlemini
Functions: awk sort tail uniq
4

This searches the Apache error_log for each of the 5 most significant Apache error levels, if any are found the date is then cut from the output in order to sort then print the most common occurrence of each error.

ifconfig -a| awk '/^wlan|^eth|^lo/ {;a=$1;FS=":"; nextline=NR+1; next}{ if (NR==nextline) { split($2,b," ")}{ if ($2 ~ /[0-9]\./) {print a,b[1]}; FS=" "}}'
2010-04-15 04:34:28
User: alf
Functions: awk ifconfig
Tags: ifconfig awk IP
5

Interfaces like lo can be omitted from the beginning, there are probably better ways of doing this, i'm a noob at awk.

[ -n "$SSH_CLIENT" ] && export DISPLAY=$(echo $SSH_CLIENT | awk '{ print $1 }'):0.0
2010-04-14 08:19:37
User: GouNiNi
Functions: awk echo export
0

In some case, you need to use remote gui on servers or simple machines and it's boring to see "cannot open display on ..." if you forgot to export your display. Juste add this line in .bashrc on remote machine. Dont forget to allow remote client on your local X server :

xhost +
tail /var/log/emerge.log | awk -F: '{print strftime("%Y%m%d %X %Z", $1),$2}'
for i in `ls ~/.mozilla/firefox/*/Cache`; do file $i | grep -i mpeg | awk '{print $1}' | sed s/.$//; done
2010-04-11 23:14:18
User: TuxOtaku
Functions: awk file grep sed
4

Ever gone to a site that has an MP3 embedded into a pesky flash player, but no download link? Well, this one-liner will yank the names of those tunes straight out of FF's cache in a nice, easy to read list. What you do with them after that is *ahem* no concern of mine. ;)

awk '{a[$1] += $2} END { for (i in a) {print i " " a[i]}}' /path/to/file
2010-04-11 02:28:09
User: italo
Functions: awk
1

Example:

cat <<EOF >/tmp/foo

5 45

80 35

4 10

80 15

80 10

4 10

5 20

EOF

awk '{a[$1] += $2} END { for (i in a) {print i "=" a[i]}}' /tmp/foo

80=60

4=20

5=65

ps aux|grep -i [p]rocessname|awk '{ print $2 }'|xargs kill
sudo awk '($9 ~ /404/)' /var/log/httpd/www.domain-access_log | awk '{print $2,$9,$7,$11}' | sort | uniq -c
2010-04-09 10:31:50
User: ninjasys
Functions: awk sort sudo uniq
Tags: log error apache
1

This command will return a full list of Error 404 pages in the given access log. The following variables have been given to awk

Hostname ($2), ERROR Code ($9), Missing Item ($7), Referrer ($11)

You can then send this into a file (>> /path/to/file), which you can open with OpenOffice as a CSV

awk '$9 == 404 {print $7}' access_log | uniq -c | sort -rn | head
2010-04-08 21:40:53
User: zlemini
Functions: awk sort uniq
8

Finds the top ten pages returning an http response code of 404 in an apache log.

iconv -f UTF16LE -t UTF-8 < SOURCE | awk 'BEGIN { RS="\r\n";} { gsub("\n", "\r"); print;}' > TARGET
2010-04-04 07:16:57
User: zhangweiwu
Functions: awk iconv
0

Sometimes you want to work on data sheets by using heirloom unic commands like cut, paste, sed, sort, wc and good old awk. But your user works on Microsoft Excel spreadsheet. The idea:

1) ask your user to save it as "Unicode Text" from Microsoft Excel and send the document to you;

2) use the given command to convert it to UTF-8 text. We carefully convert "\r\n" to local end-of-line character; and to convert "\n" (in Excel, means linebreak within the table cell") to "\r", which is carrier return but not end-of-line in Unix. If the "\n" is not replaced with "\r", for example, wc -l will report incorrect column number.

function wherepath () { for DIR in `echo $PATH | tr ":" "\n" | awk '!x[$0]++ {print $0}'`; do ls ${DIR}/$1 2>/dev/null; done }
2010-04-02 20:32:36
User: mscar
Functions: awk ls tr
Tags: find locate PATH
0

The wherepath function will search all the directories in your PATH and print a unique list of locations in the order they are first found in the PATH. (PATH often has redundant entries.) It will automatically use your 'ls' alias if you have one or you can hardcode your favorite 'ls' options in the function to get a long listing or color output for example.

Alternatives:

'whereis' only searches certain fixed locations.

'which -a' searches all the directories in your path but prints duplicates.

'locate' is great but isn't installed everywhere (and it's often too verbose).

tar pcf - home | pv -s $(du -sb home | awk '{print $1}') --rate-limit 500k | gzip > /mnt/c/home.tar.gz
2010-04-02 15:29:03
User: Sail
Functions: awk du gzip tar
1

tar directory and compress it with showing progress and Disk IO limits. Pipe Viewer can be used to view the progress of the task, Besides, he can limit the disk IO, especially useful for running Servers.

ffmpeg -f alsa -itsoffset 00:00:02.000 -ac 2 -i hw:0,0 -f x11grab -s $(xwininfo -root | grep 'geometry' | awk '{print $2;}') -r 10 -i :0.0 -sameq -f mp4 -s wvga -y intro.mp4
2010-03-31 09:33:05
User: mohan43u
Functions: awk grep
4

Yet another x11grab using ffmpeg. I also added mic input to the capturing video stream using alsa. Yet I need to find out how to capture audio which is currently playing.

mysql -e"SHOW STATUS LIKE '%uptime%'"|awk '/ptime/{ calc = $NF / 3600;print $(NF-1), calc"Hour" }'
find . -type f -iname '*.msh' -exec ls -lG {} \; | awk '{total = total + $4}END{print "scale=2;" total "/2^20"}' | bc
ffmpeg -f x11grab -s `xdpyinfo | grep 'dimensions:'|awk '{print $2}'` -r 25 -i :0.0 -sameq /tmp/out.mpg > /root/howto/capture_screen_video_ffmpeg
TR=`free|grep Mem:|awk '{print $2}'`;ps axo rss,comm,pid|awk -v tr=$TR '{proc_list[$2]+=$1;} END {for (proc in proc_list) {proc_pct=(proc_list[proc]/tr)*100; printf("%d\t%-16s\t%0.2f%\n",proc_list[proc],proc,proc_pct);}}'|sort -n |tail -n 10
2010-03-27 01:34:50
User: d34dh0r53
Functions: awk grep sort tail
3

Prints the top 10 memory consuming processes (with children and instances aggregated) sorted by total RSS and calculates the percentage of total RAM each uses. Please note that since RSS can include shared libraries it is possible for the percentages to add up to more that the total amount of RAM, but this still gives you a pretty good idea. Also note that this does not work with the mawk version of awk, but it works fine with GNU Awk which is on most Linux systems. It also does not work on OS X.

ifconfig | awk '/inet addr/ &&! /127.0.0.1/{ gsub(/addr:/,""); print $2 }'
ps -ef |grep oracle |grep pmon |awk '{print $2}' |xargs -I {} ps eww {} |grep pmon |grep -v grep |awk '{print $5 " " $6 " " $0}' |sed 's/\(S*\) \(S*\) .*ORACLE_HOME/\1 \2/g' |cut -f1,2,3 -d" "
2010-03-23 20:36:59
User: brianmuckian
Functions: awk cut grep ps sed xargs
0

get a list of currently running oracle dbs (identified by the pmon process)

show the executable that spawned the process and

show the ORACLE_HOME relative to the environment within which the process is running

tailored to AIX (sed on linux behaves...differently)

suggestions for a better way...please.

WID=xprop -root | grep "_NET_ACTIVE_WINDOW(WINDOW)"| awk '{print $5}' xdotool windowfocus $WID xdotool key ctrl+shift+t wmctrl -i -a $WID
2010-03-22 08:34:24
User: drunkcoder
Functions: awk grep
-1

open a new tab in active gnome-terminal instance

needs sudo apt-get install xdotool

and sudo apt-get install wmctrl

ls -l | awk '{if (NR % 5 == 0) print "-- COMMIT --"; print}'
alias busy='my_file=$(find /usr/include -type f | sort -R | head -n 1); my_len=$(wc -l $my_file | awk "{print $1}"); let "r = $RANDOM % $my_len" 2>/dev/null; vim +$r $my_file'
2010-03-09 21:48:41
User: busybee
Functions: alias awk find head sort vim wc
22

This makes an alias for a command named 'busy'. The 'busy' command opens a random file in /usr/include to a random line with vim. Drop this in your .bash_aliases and make sure that file is initialized in your .bashrc.