Commands using awk sorted by votes

Commands using awk (1,418)

sorted by

dd with progress bar and statistics to gzipped image
This is sample output - yours may be different.
5

sudo dd if=/dev/sdc bs=4096 | pv -s `sudo mount /dev/sdc /media/sdc && du -sb /media/sdc/ |awk '{print $1}' && sudo umount /media/sdc`| sudo dd bs=4096 of=~/USB_BLACK_BACKUP.IMG

juanmi · 2013-02-06 09:29:10 7
Fetch the current human population of Earth
This is sample output - yours may be different.
5

curl -s http://www.census.gov/popclock/data/population/world | awk -F'[:,]' '{print $7}'

zlemini · 2013-07-28 00:31:30 9
Graphically show percent of mount space used

Automatically drops mount points that have non-numeric sizes (e.g. /proc). Tested in bash on Linux and AIX. Show Sample Output
This is sample output - yours may be different.
```
/                        ============= 13
/var                     ================== 18
/home                    ============================================ 44
/tmp                     == 2
/boot                    ====== 6
/dev/shm                  0
```
5

for m in `df -P | awk -F ' ' '{print $NF}' | sed -e "1d"`;do n=`df -P | grep "$m$" | awk -F ' ' '{print $5}' | cut -d% -f1`;i=0;if [[ $n =~ ^-?[0-9]+$ ]];then printf '%-25s' $m;while [ $i -lt $n ];do echo -n '=';let "i=$i+1";done;echo " $n";fi;done

drockney · 2013-07-29 20:12:39 6
Found how how much memory in kB $PID is occupying in Linux

The "proportional set size" is probably the closest representation of how much active memory a process is using in the Linux virtual memory stack. This number should also closely represent the %mem found in ps(1), htop(1), and other utilities. Show Sample Output
This is sample output - yours may be different.
```
92327
```
5

echo 0$(awk '/Pss/ {printf "+"$2}' /proc/$PID/smaps)|bc

atoponce · 2013-09-26 18:20:22 10
Find size in kilobyte of files that are deleted but still in use and therefore consumes diskspace
This is sample output - yours may be different.
5

lsof -ns | grep REG | grep deleted | awk '{s+=$7/1024} END {print s}'

atboy · 2014-10-29 13:38:49 8
Show max lengths of all fields in a pipe delimited file with header row
This is sample output - yours may be different.
5

awk -F"|" 'BEGIN {OFS="|"} NR==1 {for (b=1;b<=NF;b++) {hdr[b]=$b} } NR > 1 {for (i=1;i<=NF;i++) {if(length($i) > max[i]) max[i] = length($i)} } END {for (i=1;i <= NF;i++) print hdr[i],max[i]+0}' pipe_delimited_file.psv

galaxywatcher · 2015-04-14 23:26:53 10
Check if your desired password is already available in haveibeenpwnd database. This command uses the API provided by HIBP
This is sample output - yours may be different.
5

function hibp() { sha1=$(echo -n "$1"|sha1sum|awk '{print toupper($0)}'|tr -d '\n'); curl -s -H $'Referer: https://haveibeenpwned.com/' https://api.pwnedpasswords.com/range/$(echo -n $sha1|cut -c1-5)|grep -i $(echo -n $sha1|cut -c6-40); }

zmonkey · 2018-08-14 07:41:55 309
AWK Calculator
This is sample output - yours may be different.
5

calc(){ awk "BEGIN{ print $* }" ;}; calc "((3+(2^3)) * 34^2 / 9)-75.89"

podiiii · 2018-12-25 04:11:14 34
Network Discover in a one liner
This is sample output - yours may be different.
5

nmap -sn 192.168.1.0/24 -oG - | awk '$4=="Status:" && $5=="Up" {print $0}'|column -t

wuseman1 · 2019-04-19 17:12:00 38
Shows picture exif GPS info if any and converts coords to a decimal degree number

This oneliner uses Imagemagic's identify utility to show the exif GPS information of an image an also converts Grad/MIn/Sec representation to a decimal degree number Show Sample Output
This is sample output - yours may be different.
```
    exif:GPSAltitude: 1665000/1000
    exif:GPSAltitudeRef: 0
    exif:GPSDateStamp: 2017:10:25
    exif:GPSInfo: 3166
    exif:GPSLatitude: 47/1, 39/1, 403802/10000  (47.661217)
    exif:GPSLatitudeRef: N
    exif:GPSLongitude: 11/1, 47/1, 374272/10000  (11.793730)
    exif:GPSLongitudeRef: E
    exif:GPSMapDatum: WGS-84
    exif:GPSProcessingMethod: ASCII
    exif:GPSTimeStamp: 10/1, 32/1, 53/1
    exif:GPSVersionID: 2, 2, 0, 0
```
5

identify -verbose my_image.jpg | awk 'function cf(i){split(i,a,"/");if(length(a)==2){return a[1]/a[2]}else{return a[1]}}/GPS/{if($1~/GPSLatitude:|GPSLongitude:/){s=$0;gsub(/,/,"",$0);printf("%s (%f)\n", s, $2+cf($3)/60+cf($4)/3600)}else{print}}'

ichbins · 2022-02-20 10:17:49 322
Display summary of git commit ids and messages for a given branch

Useful when quickly looking for a commit id from a branch to use with git cherry-pick. Show Sample Output
This is sample output - yours may be different.
```
0165c83d3ec75edded1066f4a042218748f5ef56     Added escaping of keywords on single view
94da1903caec853a3fd183405f3925e2e682b240     Test change to file
299fa310ddbb532a673e35f08c1944ae51f26261     Removed swap files
```
4

git log master | awk '/commit/ {id=$2} /\s+\w+/ {print id, $0}'

root · 2009-01-28 13:32:08 87
uniq for unsorted data
This is sample output - yours may be different.
4

awk '!_[$0]++{print}'

dz · 2009-02-06 09:52:35 20
Show top running processes by the number of open filehandles they have

I think I could cut down the number of pipes here, any suggestions? Show Sample Output
This is sample output - yours may be different.
```
sten@fenris:~$ sudo lsof | awk '{print $1}' | sort | uniq -c | sort -rn | head
   3002 php5-cgi
    122 master
    116 sshd
    115 freeswitc
     82 mysqld
     60 ssh
     60 slapd
     52 lighttpd
     47 beam.smp
     43 inet_geth
```
4

lsof | awk '{print $1}' | sort | uniq -c | sort -rn | head

sud0er · 2009-02-19 19:45:09 6
count processes with status "D" uninterruptible sleep
This is sample output - yours may be different.
4

top -b -n 1 | awk '{if (NR <=7) print; else if ($8 == "D") {print; count++} } END {print "Total status D: "count}'

jcgam69 · 2009-02-19 21:14:46 14
grep apache access.log and list IP's by hits and date - sorted
This is sample output - yours may be different.
4

grep Mar/2009 /var/log/apache2/access.log | awk '{ print $1 }' | sort -n | uniq -c | sort -rn | head

allinurl · 2009-03-10 16:46:19 9
Calculating series with awk: add numbers from 1 to 100

"seq 100" outputs 1,2,..,100, separated by newlines. awk adds them up and displays the sum. "seq 1 2 11" outputs 1,3,..,11. Variations: 1+3+...+(2n-1) = n^2 seq 1 2 19 | awk '{sum+=$1} END {print sum}' # displays 100 1/2 + 1/4 + ... = 1 seq 10 | awk '{sum+=1/(2**$1)} END {print sum}' # displays 0.999023 Show Sample Output
This is sample output - yours may be different.
```
5050
```
4

seq 100 | awk '{sum+=$1} END {print sum}'

kaan · 2009-03-24 20:30:40 5
find an unused unprivileged TCP port

Some commands (such as netcat) have a port option but how can you know which ports are unused? Show Sample Output
This is sample output - yours may be different.
```
$ netstat -atn | awk ' /tcp/ {printf("%s\n",substr($4,index($4,":")+1,length($4) )) }' | sed -e "s/://g" | sort -rnu | awk '{array [$1] = $1} END {i=32768; again=1; while (again == 1) {if (array[i] == i) {i=i+1} else {print i; again=0}}}'
32768
```
4

netstat -atn | awk ' /tcp/ {printf("%s\n",substr($4,index($4,":")+1,length($4) )) }' | sed -e "s/://g" | sort -rnu | awk '{array [$1] = $1} END {i=32768; again=1; while (again == 1) {if (array[i] == i) {i=i+1} else {print i; again=0}}}'

mpb · 2009-03-27 20:38:43 8
ensure your ssh tunnel will always be up (add in crontab)

I find it ugly & sexy at the same time isn't it ?
This is sample output - yours may be different.
4

[[ $(COLUMNS=200 ps faux | awk '/grep/ {next} /ssh -N -R 4444/ {i++} END {print i}') ]] || nohup ssh -N -R 4444:localhost:22 user@relay &

j0rn · 2009-03-31 09:39:59 10
Apache memory usage
This is sample output - yours may be different.
4

ps auxf | grep httpd | grep -v grep | grep -v defunct | awk '{sum=sum+$6}; END {print sum/1024}'

empulse · 2009-04-29 17:43:52 32
Randomize lines in a file

This appends a random number as a first filed of all lines in SOMEFILE then sorts by the first column and finally cuts of the random numbers.
This is sample output - yours may be different.
4

awk 'BEGIN{srand()}{print rand(),$0}' SOMEFILE | sort -n | cut -d ' ' -f2-

axelabs · 2009-05-29 01:20:50 11
gets all files committed to svn by a particular user since a particular date

just change the date following the -r flag, and/or the user name in the user== conditional statement, and substitute yms_web with the name of your module
This is sample output - yours may be different.
4

svn log -v -r{2009-05-21}:HEAD | awk '/^r[0-9]+ / {user=$3} /yms_web/ {if (user=="george") {print $2}}' | sort | uniq

jemptymethod · 2009-06-05 14:07:28 11
Print line immediately before a matching regex.

Use this if you don't have access to GNU grep's -B option.
This is sample output - yours may be different.
4

awk '/regex/{print x};{x=$0}'

res0nat0r · 2009-06-26 19:29:54 10
Hear the mice moving

Beeps on mouse's every move. Bear in mind that, at least on Ubuntu, /dev/input/mice can be read only by root.
This is sample output - yours may be different.
4

while true; do beep -l66 -f`head -c2 /dev/input/mice|hexdump -d|awk 'NR==1{print $2%10000}'`; done

5z474n · 2009-07-11 12:01:27 22
Retrieve top ip threats from http://isc.sans.org/sources.html and add them into iptables output chain.

Retrieve top ip threats from http://isc.sans.org/sources.html and add them into iptables output chain.
This is sample output - yours may be different.
4

curl -s http://isc.sans.org/sources.html|grep "ipinfo.html"|awk -F"ip=" {'print $2'}|awk -F"\"" {'print $1'}|xargs -n1 sudo iptables -A OUTPUT -j DROP -d > 2&>1

din7 · 2009-08-03 20:18:41 12
print file without duplicated lines usind awk

show file withou duplicated lines
This is sample output - yours may be different.
4

awk '!($0 in a) {a[$0];print}' file

voyeg3r · 2009-08-23 12:54:41 6
‹ First < 5 6 7 8 9 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

memory usage

Transfer Entire recursive from one host to another. Only copies files that are newer or do not exist

From opposite host To copy remote to local rsync -aE -e "ssh -pPortnumber" user@hostA:directory target_dir

Which processes are listening on a specific port (e.g. port 80)

swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Key binding to search commandlinefu.com

This is a simple bash function and a key binding that uses commandlinefu's simple and easy search API. It prompts for a search term, then it uses curl to search commandline fu, and highlights the search results with less.

mplayer webcam window for screencasts

When recording screencast some people like to have the image from their webcam, so the can show something, that can't be seen on the desktop. So starting mplayer with these parameters you will have a window with no frames, borders whatsoever, and selecting the window a hitting the "F" key you will bring it in fullscreen. if you want to position the frame somewhere else, you could play with the --geomeptry option where 100%:100% mean bottom right corner. The HEIGHT and WIDTH can't be changed as you like, since the most webcams support specified dimensions, so you would have to play with it to see what is supported

Find out how old a web page is

I used to use the Firefox "View page info" feature a lot to determine how stale the web page I was looking at was. Now that I use mostly Chrome I miss that feature, so here is a command line alternative using wget. The -S says to display the server response, the --spider says to not download any files/pages, just fetch the header. The output goes to stderr, so to grep it you use 2>&1 to combine the stderr stream with stdout, the pipe that to grep for Last-Modified. You can use curl instead if you have it installed, like this: $ curl --head -s http://osswin.sourceforge.net | grep Mod

Delete a file/directory walking subdirectories (bash4 or zsh)

ps with parent/child process tree

Shows a tree view of parent to child processes in the output of ps (linux). Similar output can be achieved with pstree (also linux) or ptree (Solaris).

Manipulate the metadata when the photo was taken, this will shift with +15hours + 30min

Fix borked character coding in a tty.

Often you find some tty programs are messed up and confused about character encoding - 'man' is a common problem and sometimes displays weird characters for apostrophes, hyphens etc etc. Another class of programs that suffer from this are those that try to use the line drawing characters - eg RedHat's tty system admin functions such as system-config-firewall-tui system-config-network-tui etc. Adding 'LC_ALL=C' fixes most of these problems (as long as you want English! Perhaps speakers of other languages can add a comment here). For bonus points, I've added the '-c' option to the man command so that it ignores it's cache and re-computes the man page using the C locale.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using awk

Commands using awk (1,418) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using awk (1,418)

sorted by