Commands tagged awk sorted by votes

Commands tagged awk (348)

sorted by

Convert number of bytes to human readable filesize

Converts a number of bytes provided as input, to a human readable number. Show Sample Output
This is sample output - yours may be different.
```
$ human_filesize 1234567890
1.15 Gb
```
2

human_filesize() { awk -v sum="$1" ' BEGIN {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; } '}

ArtBIT · 2011-12-02 18:21:20 3
Get the date for the last Saturday of a given month

If your locale has Monday as the first day of the week, like mine in the UK, change the two $7 into $6 Show Sample Output
This is sample output - yours may be different.
```
28
```
2

cal 04 2012 | awk '{ $7 && X=$7 } END { print X }'

flatcap · 2012-05-06 23:43:21 40
Count the number of pages of all PDFs in current directory and all subdirs, recursively
This is sample output - yours may be different.
2

find . -name "*.pdf" -exec pdftk {} dump_data output \; | grep NumberOfPages | awk '{s+=$2} END {print s}'

sucotronic · 2013-01-09 15:26:45 43
Don't like the cut command? Tired of typing awk '{print $xxx}', try this

Show Sample Output
This is sample output - yours may be different.
```
# ap () { awk '{print $'$1'}'; }
# echo hi there | ap 2
there
```
2

awp () { awk '{print $'$1'}'; }

joeheyming · 2014-08-15 16:21:49 9
drop first column of output by piping to this
This is sample output - yours may be different.
2

awk '{for(i=2;i<=NF;i++) printf("%s%s",$i,(i!=NF)?OFS:ORS)}'

snaguber · 2015-05-26 23:14:39 12
Monitor memory fine-grained usage (e.g. firefox)

Sometimes top/htop don't give the fine-grained detail on memory usage you might need. Sum up the exact memory types you want
This is sample output - yours may be different.
2

watch "awk '/Rss/{sum += \$2; } END{print sum, \"kB\"}' < /proc/$(pidof firefox)/smaps"

gumnos · 2015-09-19 00:36:34 18
Calculates the number of physical cores considering HyperThreading in AWK

Check whether hyperthreading is enabled or not. a better solution as nproc should work on all OS with awk Show Sample Output
This is sample output - yours may be different.
```
# example: 4xCPU 24xHT Cores
48
```
2

awk -F: '/^core id/ && !P[$2] { CORES++; P[$2]=1 }; /^physical id/ && !N[$2] { CPUs++; N[$2]=1 }; END { print CPUs*CORES }' /proc/cpuinfo

emphazer · 2018-05-14 14:17:33 160
Ultra fast public IP address lookup using Cloudflare's 1.1.1.1
This is sample output - yours may be different.
2

curl -fSs https://1.1.1.1/cdn-cgi/trace | awk -F= '/ip/ { print $2 }'

skhaz · 2019-09-05 19:31:25 247
Print nic name of current connection
This is sample output - yours may be different.
2

nmcli -g name,type connection show --active|awk -F: '/ethernet|wireless/ { print $1 }'

wuseman1 · 2022-09-22 20:16:21 662
urldecode with AWK

Fast and simple awk urldecoder! Note: Parameter -n is specific to GNU awk Show Sample Output
This is sample output - yours may be different.
```
echo -e  "asdf%2Fxx%2Cx\n%2F%2Cxx"  | awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..
asdf/xx,x
/,xx
```
2

awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..

emphazer · 2023-03-30 15:05:58 319
Display duplicated lines in a file

Displays the duplicated lines in a file and their occuring frequency.
This is sample output - yours may be different.
1

cat file.txt | sort | uniq -dc

Vadi · 2009-03-21 18:15:14 7
sync svn working copy and remote repository (auto adding new files)

Lists the local files that are not present in the remote repository (lines beginning with ?) and add them. Show Sample Output
This is sample output - yours may be different.
```
$ svn status | grep '^?'
?      INFO
?      src/svn-commit.tmp
?      src/game/Game.java
?      src/game/character/Hero.java

$ svn status | grep '^?' | awk '{ print $2; }' | xargs svn add
$ svn ci

A      INFO
A      src/svn-commit.tmp
A      src/game/Game.java
A      src/game/character/Hero.java
```
1

svn status | grep '^?' | awk '{ print $2; }' | xargs svn add

unixmonkey4200 · 2009-04-10 21:55:37 9
Uncompress a CSS file

Ever compress a file for the web by replacing all newline characters with nothing so it makes one nice big blob? It is a great idea, however what about when you want to edit that file? ...Serious pain in the butt. I ran into this today in that my only copy of a CSS file was "compressed" with no newlines. I whipped this up and it converted back into nice human readable CSS :-) It could be nicer, but it does the job.
This is sample output - yours may be different.
1

cat somefile.css | awk '{gsub(/{|}|;/,"&\n"); print}' >> uncompressed.css

lrvick · 2009-06-02 15:51:51 37
Puts every word from a file into a new line

Basically it creates a typical word list file from any normal text.
This is sample output - yours may be different.
1

awk '{c=split($0, s); for(n=1; n<=c; ++n) print s[n] }' INPUT_FILE > OUTPUT_FILE

agony · 2009-07-06 06:10:21 8
Remove newlines from output

?Cat and grep? You can use only grep ("grep \. filename"). Better option is awk.
This is sample output - yours may be different.
1

awk /./ filename

point_to_null · 2009-08-09 02:04:46 4
Sum columns from CSV column $COL

More of the same but with more elaborate perl-fu :-)
This is sample output - yours may be different.
1

perl -F',' -ane '$a += $F[3]; END { print $a }' test.csv

coffeeaddict_nl · 2009-08-11 15:08:58 3
Checks apache's access_log file, strips the search queries and shoves them up your e-mail

as unixmonkey7109 pointed out, first awk parse replaces three steps.
This is sample output - yours may be different.
1

awk '/q=/{print $11}' /var/log/httpd/access_log.4 | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1

isma · 2009-11-22 17:13:48 69
random xkcd comic as xml
This is sample output - yours may be different.
1

curl -sL 'dynamic.xkcd.com/comic/random/' | awk -F\" '/^<img/{printf("<?xml version=\"1.0\"?>\n<xkcd>\n<item>\n <title>%s</title>\n <comment>%s</comment>\n <image>%s</image>\n</item>\n</xkcd>\n", $6, $4, $2)}'

putnamhill · 2009-12-09 15:02:32 4
ShadyURL via CLI
This is sample output - yours may be different.
1

SITE="www.google.com"; curl --silent "http://www.shadyurl.com/create.php?myUrl=$SITE&shorten=on" | awk -F\' '/is now/{print $6}'

allrightname · 2010-02-19 14:21:02 6
Extract title from HTML files

previous version leaves lots of blank lines
This is sample output - yours may be different.
1

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' | sed '/^$/d' > file.html

tamouse · 2010-04-20 13:27:47 4
Extract title from HTML files

not the best, uses 4 pipes!
This is sample output - yours may be different.
1

tr -d "\n\r" | grep -ioEm1 "<title[^>]*>[^<]*</title" | cut -f2 -d\> | cut -f1 -d\<

bandie91 · 2010-04-20 18:55:24 3
Count TCP States From Netstat

Counts TCP states from Netstat and displays in an ordered list. Show Sample Output
This is sample output - yours may be different.
```
9 ESTABLISHED
9 LISTEN
1 SYN_SENT
```
1

netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c

Kered557 · 2010-05-06 17:04:37 4
Show DeviceMapper names for LVM Volumes (to disambiguate iostat logs, etc)

Emits the device names which will be printed by iostat for an LVM volume; doesn't show the names for the underlying devices when snapshots are being used (the -cow and -real devices in /dev/mapper) Show Sample Output
This is sample output - yours may be different.
```
dm-0 /dev/storage/root
dm-1 /dev/storage/swap_1
```
1

sudo lvdisplay |awk '/LV Name/{blockdev=$3} /Block device/{bdid=$3; sub("[0-9]*:","dm-",bdid); print bdid,blockdev;}'

unixmonkey10270 · 2010-06-10 11:50:31 3
Add all unversioned files to svn

No need for grep, let awk do the match. This will not behave properly if the filenames contains whitespace, which is awk's default field separator.
This is sample output - yours may be different.
1

svn st | awk '{if ($1 ~ "?") print $2}' | xargs svn add

sciurus · 2010-06-19 03:07:26 6
Edit a file in vim (at the first error) if it is not well formed xml.

Validate a file using xmllint. If there are parser errors, edit the file in vim at the line of the first error.
This is sample output - yours may be different.
1

vimlint(){ eval $(xmllint --noout "$1" 2>&1 | awk -F: '/parser error/{print "vim \""$1"\" +"$2; exit}'); }

putnamhill · 2010-06-23 15:55:02 3
‹ First < 3 4 5 6 7 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Get the Nth argument of the last command (handling spaces correctly)

Bash's history expansion character, "!", has many features, including "!:" for choosing a specific argument (or range of arguments) from the history. The gist is any number after !: is the number of the argument you want, with !:1 being the first argument and !:0 being the command. See the sample output for a few examples. For full details search for "^HISTORY EXPANSION" in the bash(1) man page. Note that this version improves on the previous function in that it handles arguments that include whitespace correctly.

kill ip connection

needed; apt-get install tcpkill

list files recursively by size

Print a monthly calendar with today's date highlighted

The cal command is handy, but sometimes you want to quickly see today's date highlighted. That's why I came up with this quick command. Much like http://www.commandlinefu.com/commands/view/1342/show-this-months-calendar-with-todays-date-highlighted but cleaner and more succinct.

Find all clients connected to HTTP or HTTPS ports

It finds, specifically, the connections to the HTTP and HTTPS ports as source ports. You can check for destination ports as well.

Which processes are listening on a specific port (e.g. port 80)

swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Which processes are listening on a specific port (e.g. port 80)

swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Check if *hardware* is 32bit or 64bit

This command tell you if your hardware is 32 or 64 bits even if you install a 32bits OS on a 64 bits hardware. If your distro don't support the -q switch, try doing : $ grep &>/dev/null '\' /proc/cpuinfo && echo 64 bits || echo 32 bits

find filenames and directory names that doesn't conform ISO 9660 level 2

You might want to check what file and directory names would be renamed or chopped if you create iso 9660 level 2 image out of them. Use this command to check first.

Recursively remove all empty directories

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Commands tagged awk (348) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged awk (348)

sorted by