Commands using awk sorted by votes

Commands using awk (1,418)

sorted by

dd with progress bar and statistics to gzipped image
This is sample output - yours may be different.
5

sudo dd if=/dev/sdc bs=4096 | pv -s `sudo mount /dev/sdc /media/sdc && du -sb /media/sdc/ |awk '{print $1}' && sudo umount /media/sdc`| sudo dd bs=4096 of=~/USB_BLACK_BACKUP.IMG

juanmi · 2013-02-06 09:29:10 7
Fetch the current human population of Earth
This is sample output - yours may be different.
5

curl -s http://www.census.gov/popclock/data/population/world | awk -F'[:,]' '{print $7}'

zlemini · 2013-07-28 00:31:30 9
Graphically show percent of mount space used

Automatically drops mount points that have non-numeric sizes (e.g. /proc). Tested in bash on Linux and AIX. Show Sample Output
This is sample output - yours may be different.
```
/                        ============= 13
/var                     ================== 18
/home                    ============================================ 44
/tmp                     == 2
/boot                    ====== 6
/dev/shm                  0
```
5

for m in `df -P | awk -F ' ' '{print $NF}' | sed -e "1d"`;do n=`df -P | grep "$m$" | awk -F ' ' '{print $5}' | cut -d% -f1`;i=0;if [[ $n =~ ^-?[0-9]+$ ]];then printf '%-25s' $m;while [ $i -lt $n ];do echo -n '=';let "i=$i+1";done;echo " $n";fi;done

drockney · 2013-07-29 20:12:39 6
Found how how much memory in kB $PID is occupying in Linux

The "proportional set size" is probably the closest representation of how much active memory a process is using in the Linux virtual memory stack. This number should also closely represent the %mem found in ps(1), htop(1), and other utilities. Show Sample Output
This is sample output - yours may be different.
```
92327
```
5

echo 0$(awk '/Pss/ {printf "+"$2}' /proc/$PID/smaps)|bc

atoponce · 2013-09-26 18:20:22 10
Find size in kilobyte of files that are deleted but still in use and therefore consumes diskspace
This is sample output - yours may be different.
5

lsof -ns | grep REG | grep deleted | awk '{s+=$7/1024} END {print s}'

atboy · 2014-10-29 13:38:49 8
Show max lengths of all fields in a pipe delimited file with header row
This is sample output - yours may be different.
5

awk -F"|" 'BEGIN {OFS="|"} NR==1 {for (b=1;b<=NF;b++) {hdr[b]=$b} } NR > 1 {for (i=1;i<=NF;i++) {if(length($i) > max[i]) max[i] = length($i)} } END {for (i=1;i <= NF;i++) print hdr[i],max[i]+0}' pipe_delimited_file.psv

galaxywatcher · 2015-04-14 23:26:53 10
Check if your desired password is already available in haveibeenpwnd database. This command uses the API provided by HIBP
This is sample output - yours may be different.
5

function hibp() { sha1=$(echo -n "$1"|sha1sum|awk '{print toupper($0)}'|tr -d '\n'); curl -s -H $'Referer: https://haveibeenpwned.com/' https://api.pwnedpasswords.com/range/$(echo -n $sha1|cut -c1-5)|grep -i $(echo -n $sha1|cut -c6-40); }

zmonkey · 2018-08-14 07:41:55 312
AWK Calculator
This is sample output - yours may be different.
5

calc(){ awk "BEGIN{ print $* }" ;}; calc "((3+(2^3)) * 34^2 / 9)-75.89"

podiiii · 2018-12-25 04:11:14 34
Network Discover in a one liner
This is sample output - yours may be different.
5

nmap -sn 192.168.1.0/24 -oG - | awk '$4=="Status:" && $5=="Up" {print $0}'|column -t

wuseman1 · 2019-04-19 17:12:00 38
Shows picture exif GPS info if any and converts coords to a decimal degree number

This oneliner uses Imagemagic's identify utility to show the exif GPS information of an image an also converts Grad/MIn/Sec representation to a decimal degree number Show Sample Output
This is sample output - yours may be different.
```
    exif:GPSAltitude: 1665000/1000
    exif:GPSAltitudeRef: 0
    exif:GPSDateStamp: 2017:10:25
    exif:GPSInfo: 3166
    exif:GPSLatitude: 47/1, 39/1, 403802/10000  (47.661217)
    exif:GPSLatitudeRef: N
    exif:GPSLongitude: 11/1, 47/1, 374272/10000  (11.793730)
    exif:GPSLongitudeRef: E
    exif:GPSMapDatum: WGS-84
    exif:GPSProcessingMethod: ASCII
    exif:GPSTimeStamp: 10/1, 32/1, 53/1
    exif:GPSVersionID: 2, 2, 0, 0
```
5

identify -verbose my_image.jpg | awk 'function cf(i){split(i,a,"/");if(length(a)==2){return a[1]/a[2]}else{return a[1]}}/GPS/{if($1~/GPSLatitude:|GPSLongitude:/){s=$0;gsub(/,/,"",$0);printf("%s (%f)\n", s, $2+cf($3)/60+cf($4)/3600)}else{print}}'

ichbins · 2022-02-20 10:17:49 322
Display summary of git commit ids and messages for a given branch

Useful when quickly looking for a commit id from a branch to use with git cherry-pick. Show Sample Output
This is sample output - yours may be different.
```
0165c83d3ec75edded1066f4a042218748f5ef56     Added escaping of keywords on single view
94da1903caec853a3fd183405f3925e2e682b240     Test change to file
299fa310ddbb532a673e35f08c1944ae51f26261     Removed swap files
```
4

git log master | awk '/commit/ {id=$2} /\s+\w+/ {print id, $0}'

root · 2009-01-28 13:32:08 87
uniq for unsorted data
This is sample output - yours may be different.
4

awk '!_[$0]++{print}'

dz · 2009-02-06 09:52:35 20
Show top running processes by the number of open filehandles they have

I think I could cut down the number of pipes here, any suggestions? Show Sample Output
This is sample output - yours may be different.
```
sten@fenris:~$ sudo lsof | awk '{print $1}' | sort | uniq -c | sort -rn | head
   3002 php5-cgi
    122 master
    116 sshd
    115 freeswitc
     82 mysqld
     60 ssh
     60 slapd
     52 lighttpd
     47 beam.smp
     43 inet_geth
```
4

lsof | awk '{print $1}' | sort | uniq -c | sort -rn | head

sud0er · 2009-02-19 19:45:09 6
count processes with status "D" uninterruptible sleep
This is sample output - yours may be different.
4

top -b -n 1 | awk '{if (NR <=7) print; else if ($8 == "D") {print; count++} } END {print "Total status D: "count}'

jcgam69 · 2009-02-19 21:14:46 14
grep apache access.log and list IP's by hits and date - sorted
This is sample output - yours may be different.
4

grep Mar/2009 /var/log/apache2/access.log | awk '{ print $1 }' | sort -n | uniq -c | sort -rn | head

allinurl · 2009-03-10 16:46:19 9
Calculating series with awk: add numbers from 1 to 100

"seq 100" outputs 1,2,..,100, separated by newlines. awk adds them up and displays the sum. "seq 1 2 11" outputs 1,3,..,11. Variations: 1+3+...+(2n-1) = n^2 seq 1 2 19 | awk '{sum+=$1} END {print sum}' # displays 100 1/2 + 1/4 + ... = 1 seq 10 | awk '{sum+=1/(2**$1)} END {print sum}' # displays 0.999023 Show Sample Output
This is sample output - yours may be different.
```
5050
```
4

seq 100 | awk '{sum+=$1} END {print sum}'

kaan · 2009-03-24 20:30:40 5
find an unused unprivileged TCP port

Some commands (such as netcat) have a port option but how can you know which ports are unused? Show Sample Output
This is sample output - yours may be different.
```
$ netstat -atn | awk ' /tcp/ {printf("%s\n",substr($4,index($4,":")+1,length($4) )) }' | sed -e "s/://g" | sort -rnu | awk '{array [$1] = $1} END {i=32768; again=1; while (again == 1) {if (array[i] == i) {i=i+1} else {print i; again=0}}}'
32768
```
4

netstat -atn | awk ' /tcp/ {printf("%s\n",substr($4,index($4,":")+1,length($4) )) }' | sed -e "s/://g" | sort -rnu | awk '{array [$1] = $1} END {i=32768; again=1; while (again == 1) {if (array[i] == i) {i=i+1} else {print i; again=0}}}'

mpb · 2009-03-27 20:38:43 8
ensure your ssh tunnel will always be up (add in crontab)

I find it ugly & sexy at the same time isn't it ?
This is sample output - yours may be different.
4

[[ $(COLUMNS=200 ps faux | awk '/grep/ {next} /ssh -N -R 4444/ {i++} END {print i}') ]] || nohup ssh -N -R 4444:localhost:22 user@relay &

j0rn · 2009-03-31 09:39:59 10
Apache memory usage
This is sample output - yours may be different.
4

ps auxf | grep httpd | grep -v grep | grep -v defunct | awk '{sum=sum+$6}; END {print sum/1024}'

empulse · 2009-04-29 17:43:52 32
Randomize lines in a file

This appends a random number as a first filed of all lines in SOMEFILE then sorts by the first column and finally cuts of the random numbers.
This is sample output - yours may be different.
4

awk 'BEGIN{srand()}{print rand(),$0}' SOMEFILE | sort -n | cut -d ' ' -f2-

axelabs · 2009-05-29 01:20:50 11
gets all files committed to svn by a particular user since a particular date

just change the date following the -r flag, and/or the user name in the user== conditional statement, and substitute yms_web with the name of your module
This is sample output - yours may be different.
4

svn log -v -r{2009-05-21}:HEAD | awk '/^r[0-9]+ / {user=$3} /yms_web/ {if (user=="george") {print $2}}' | sort | uniq

jemptymethod · 2009-06-05 14:07:28 11
Print line immediately before a matching regex.

Use this if you don't have access to GNU grep's -B option.
This is sample output - yours may be different.
4

awk '/regex/{print x};{x=$0}'

res0nat0r · 2009-06-26 19:29:54 10
Hear the mice moving

Beeps on mouse's every move. Bear in mind that, at least on Ubuntu, /dev/input/mice can be read only by root.
This is sample output - yours may be different.
4

while true; do beep -l66 -f`head -c2 /dev/input/mice|hexdump -d|awk 'NR==1{print $2%10000}'`; done

5z474n · 2009-07-11 12:01:27 22
Retrieve top ip threats from http://isc.sans.org/sources.html and add them into iptables output chain.

Retrieve top ip threats from http://isc.sans.org/sources.html and add them into iptables output chain.
This is sample output - yours may be different.
4

curl -s http://isc.sans.org/sources.html|grep "ipinfo.html"|awk -F"ip=" {'print $2'}|awk -F"\"" {'print $1'}|xargs -n1 sudo iptables -A OUTPUT -j DROP -d > 2&>1

din7 · 2009-08-03 20:18:41 12
print file without duplicated lines usind awk

show file withou duplicated lines
This is sample output - yours may be different.
4

awk '!($0 in a) {a[$0];print}' file

voyeg3r · 2009-08-23 12:54:41 6
‹ First < 5 6 7 8 9 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Two command output

Summarize established connections after netstat output. Using tee and /dev/stderr you can send one command output to terminal before executing wc so you can summarize at the bottom of the output.

Display the definition of a shell function

Display the code of a previously defined shell function.

Split a tarball into multiple parts

Create a tar file in multiple parts if it's to large for a single disk, your filesystem, etc. Rejoin later with `cat .tar.*|tar xf -`

check open ports without netstat or lsof

Query Wikipedia via console over DNS

Query Wikipedia by issuing a DNS query for a TXT record. The TXT record will also include a short URL to the complete corresponding Wikipedia entry.You can also write a little shell script like: $ $ cat wikisole.sh $ #!/bin/sh $ dig +short txt ${1}.wp.dg.cx and run it like $ ./wikisole.sh unix were your first option ($1) will be used as search term.

Run a command that has been aliased without the alias

Most distributions alias cp to 'cp -i', which means when you attempt to copy into a directory that already contains the file, cp will prompt to overwrite. A great default to have, but when you mean to overwrite thousands of files, you don't want to sit there hitting [y] then [enter] thousands of times. Enter the backslash. It runs the command unaliased, so as in the example, cp will happily overwrite existing files much in the way mv works.

create an emergency swapfile when the existing swap space is getting tight

Create a temporary file that acts as swap space. In this example it's a 1GB file at the root of the file system. This additional capacity is added to the existing swap space.

check open ports without netstat or lsof

Generate map of your hardware

find packages installed from e.g. sid which are newer than those available from e.g. testing when sid is no longer present as a source repo

This is useful if you add sid, install some packages, then remove sid and want to work out which packages you installed from sid that should be removed (e.g. before an upgrade to the new stable). Alternatively you can think of this as "find installed packages that can no longer be installed."

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using awk

Commands using awk (1,418) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands using awk (1,418)

sorted by