Commands tagged awk sorted by votes

Commands tagged awk (348)

sorted by

Convert number of bytes to human readable filesize

Converts a number of bytes provided as input, to a human readable number. Show Sample Output
This is sample output - yours may be different.
```
$ human_filesize 1234567890
1.15 Gb
```
2

human_filesize() { awk -v sum="$1" ' BEGIN {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; } '}

ArtBIT · 2011-12-02 18:21:20 3
Get the date for the last Saturday of a given month

If your locale has Monday as the first day of the week, like mine in the UK, change the two $7 into $6 Show Sample Output
This is sample output - yours may be different.
```
28
```
2

cal 04 2012 | awk '{ $7 && X=$7 } END { print X }'

flatcap · 2012-05-06 23:43:21 40
Count the number of pages of all PDFs in current directory and all subdirs, recursively
This is sample output - yours may be different.
2

find . -name "*.pdf" -exec pdftk {} dump_data output \; | grep NumberOfPages | awk '{s+=$2} END {print s}'

sucotronic · 2013-01-09 15:26:45 43
Don't like the cut command? Tired of typing awk '{print $xxx}', try this

Show Sample Output
This is sample output - yours may be different.
```
# ap () { awk '{print $'$1'}'; }
# echo hi there | ap 2
there
```
2

awp () { awk '{print $'$1'}'; }

joeheyming · 2014-08-15 16:21:49 9
drop first column of output by piping to this
This is sample output - yours may be different.
2

awk '{for(i=2;i<=NF;i++) printf("%s%s",$i,(i!=NF)?OFS:ORS)}'

snaguber · 2015-05-26 23:14:39 12
Monitor memory fine-grained usage (e.g. firefox)

Sometimes top/htop don't give the fine-grained detail on memory usage you might need. Sum up the exact memory types you want
This is sample output - yours may be different.
2

watch "awk '/Rss/{sum += \$2; } END{print sum, \"kB\"}' < /proc/$(pidof firefox)/smaps"

gumnos · 2015-09-19 00:36:34 18
Calculates the number of physical cores considering HyperThreading in AWK

Check whether hyperthreading is enabled or not. a better solution as nproc should work on all OS with awk Show Sample Output
This is sample output - yours may be different.
```
# example: 4xCPU 24xHT Cores
48
```
2

awk -F: '/^core id/ && !P[$2] { CORES++; P[$2]=1 }; /^physical id/ && !N[$2] { CPUs++; N[$2]=1 }; END { print CPUs*CORES }' /proc/cpuinfo

emphazer · 2018-05-14 14:17:33 154
Ultra fast public IP address lookup using Cloudflare's 1.1.1.1
This is sample output - yours may be different.
2

curl -fSs https://1.1.1.1/cdn-cgi/trace | awk -F= '/ip/ { print $2 }'

skhaz · 2019-09-05 19:31:25 246
Print nic name of current connection
This is sample output - yours may be different.
2

nmcli -g name,type connection show --active|awk -F: '/ethernet|wireless/ { print $1 }'

wuseman1 · 2022-09-22 20:16:21 662
urldecode with AWK

Fast and simple awk urldecoder! Note: Parameter -n is specific to GNU awk Show Sample Output
This is sample output - yours may be different.
```
echo -e  "asdf%2Fxx%2Cx\n%2F%2Cxx"  | awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..
asdf/xx,x
/,xx
```
2

awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..

emphazer · 2023-03-30 15:05:58 318
Display duplicated lines in a file

Displays the duplicated lines in a file and their occuring frequency.
This is sample output - yours may be different.
1

cat file.txt | sort | uniq -dc

Vadi · 2009-03-21 18:15:14 7
sync svn working copy and remote repository (auto adding new files)

Lists the local files that are not present in the remote repository (lines beginning with ?) and add them. Show Sample Output
This is sample output - yours may be different.
```
$ svn status | grep '^?'
?      INFO
?      src/svn-commit.tmp
?      src/game/Game.java
?      src/game/character/Hero.java

$ svn status | grep '^?' | awk '{ print $2; }' | xargs svn add
$ svn ci

A      INFO
A      src/svn-commit.tmp
A      src/game/Game.java
A      src/game/character/Hero.java
```
1

svn status | grep '^?' | awk '{ print $2; }' | xargs svn add

unixmonkey4200 · 2009-04-10 21:55:37 9
Uncompress a CSS file

Ever compress a file for the web by replacing all newline characters with nothing so it makes one nice big blob? It is a great idea, however what about when you want to edit that file? ...Serious pain in the butt. I ran into this today in that my only copy of a CSS file was "compressed" with no newlines. I whipped this up and it converted back into nice human readable CSS :-) It could be nicer, but it does the job.
This is sample output - yours may be different.
1

cat somefile.css | awk '{gsub(/{|}|;/,"&\n"); print}' >> uncompressed.css

lrvick · 2009-06-02 15:51:51 37
Puts every word from a file into a new line

Basically it creates a typical word list file from any normal text.
This is sample output - yours may be different.
1

awk '{c=split($0, s); for(n=1; n<=c; ++n) print s[n] }' INPUT_FILE > OUTPUT_FILE

agony · 2009-07-06 06:10:21 8
Remove newlines from output

?Cat and grep? You can use only grep ("grep \. filename"). Better option is awk.
This is sample output - yours may be different.
1

awk /./ filename

point_to_null · 2009-08-09 02:04:46 4
Sum columns from CSV column $COL

More of the same but with more elaborate perl-fu :-)
This is sample output - yours may be different.
1

perl -F',' -ane '$a += $F[3]; END { print $a }' test.csv

coffeeaddict_nl · 2009-08-11 15:08:58 3
Checks apache's access_log file, strips the search queries and shoves them up your e-mail

as unixmonkey7109 pointed out, first awk parse replaces three steps.
This is sample output - yours may be different.
1

awk '/q=/{print $11}' /var/log/httpd/access_log.4 | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1

isma · 2009-11-22 17:13:48 69
random xkcd comic as xml
This is sample output - yours may be different.
1

curl -sL 'dynamic.xkcd.com/comic/random/' | awk -F\" '/^<img/{printf("<?xml version=\"1.0\"?>\n<xkcd>\n<item>\n <title>%s</title>\n <comment>%s</comment>\n <image>%s</image>\n</item>\n</xkcd>\n", $6, $4, $2)}'

putnamhill · 2009-12-09 15:02:32 4
ShadyURL via CLI
This is sample output - yours may be different.
1

SITE="www.google.com"; curl --silent "http://www.shadyurl.com/create.php?myUrl=$SITE&shorten=on" | awk -F\' '/is now/{print $6}'

allrightname · 2010-02-19 14:21:02 6
Extract title from HTML files

previous version leaves lots of blank lines
This is sample output - yours may be different.
1

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' | sed '/^$/d' > file.html

tamouse · 2010-04-20 13:27:47 4
Extract title from HTML files

not the best, uses 4 pipes!
This is sample output - yours may be different.
1

tr -d "\n\r" | grep -ioEm1 "<title[^>]*>[^<]*</title" | cut -f2 -d\> | cut -f1 -d\<

bandie91 · 2010-04-20 18:55:24 3
Count TCP States From Netstat

Counts TCP states from Netstat and displays in an ordered list. Show Sample Output
This is sample output - yours may be different.
```
9 ESTABLISHED
9 LISTEN
1 SYN_SENT
```
1

netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c

Kered557 · 2010-05-06 17:04:37 4
Show DeviceMapper names for LVM Volumes (to disambiguate iostat logs, etc)

Emits the device names which will be printed by iostat for an LVM volume; doesn't show the names for the underlying devices when snapshots are being used (the -cow and -real devices in /dev/mapper) Show Sample Output
This is sample output - yours may be different.
```
dm-0 /dev/storage/root
dm-1 /dev/storage/swap_1
```
1

sudo lvdisplay |awk '/LV Name/{blockdev=$3} /Block device/{bdid=$3; sub("[0-9]*:","dm-",bdid); print bdid,blockdev;}'

unixmonkey10270 · 2010-06-10 11:50:31 3
Add all unversioned files to svn

No need for grep, let awk do the match. This will not behave properly if the filenames contains whitespace, which is awk's default field separator.
This is sample output - yours may be different.
1

svn st | awk '{if ($1 ~ "?") print $2}' | xargs svn add

sciurus · 2010-06-19 03:07:26 6
Edit a file in vim (at the first error) if it is not well formed xml.

Validate a file using xmllint. If there are parser errors, edit the file in vim at the line of the first error.
This is sample output - yours may be different.
1

vimlint(){ eval $(xmllint --noout "$1" 2>&1 | awk -F: '/parser error/{print "vim \""$1"\" +"$2; exit}'); }

putnamhill · 2010-06-23 15:55:02 3
‹ First < 3 4 5 6 7 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Count the number of man pages per first character (a-z)

There once was a day I needed this info.

Wordwrap long text string using "\n"

I used this fragment with Imagemagick convert so that I can place long text strings in pictures. The "\n" gets converted to a true newline in the image. So this fragment uses fold command to wrap the line and then sed to convert newlines (and any trailing spaces on the line) to the text "\n"

Mount FileVault sparsebundle image manually (e.g.: from TimeMachine disk).

Set an alarm to wake up

Sleep 5h and plays the song as a wake up alarm

Install pip with Proxy

Installs pip packages defining a proxy

For a $FILE, extracts the path, filename, filename without extension and extension.

Useful for use in other scripts for renaming, testing for extensions, etc.

Clear terminal Screen

convert unixtime to human-readable with awk

- convert unixtime to human-readable with awk - useful to read logfiles with unix-timestamps, f.e. squid-log: sudo tail -f /var/log/squid3/access.log | awk '{ print strftime("%c ", $1) $0; }

Generate an XKCD #936 style 4 word password

4 random words are better than one obfuscated word http://xkcd.com/936/

Find all active ip's in a subnet

nmap for windows and other platforms is available on developer's site: http://nmap.org/download.html nmap is robust tool with many options and has various output modes - is the best (imho) tool out there.. from nmap 5.21 man page: -oN/-oX/-oS/-oG : Output scan in normal, XML, s|

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Commands tagged awk (348) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged awk (348)

sorted by