Commands tagged awk sorted by votes

Commands tagged awk (348)

sorted by

Convert number of bytes to human readable filesize

Converts a number of bytes provided as input, to a human readable number. Show Sample Output
This is sample output - yours may be different.
```
$ human_filesize 1234567890
1.15 Gb
```
2

human_filesize() { awk -v sum="$1" ' BEGIN {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; } '}

ArtBIT · 2011-12-02 18:21:20 3
Get the date for the last Saturday of a given month

If your locale has Monday as the first day of the week, like mine in the UK, change the two $7 into $6 Show Sample Output
This is sample output - yours may be different.
```
28
```
2

cal 04 2012 | awk '{ $7 && X=$7 } END { print X }'

flatcap · 2012-05-06 23:43:21 40
Count the number of pages of all PDFs in current directory and all subdirs, recursively
This is sample output - yours may be different.
2

find . -name "*.pdf" -exec pdftk {} dump_data output \; | grep NumberOfPages | awk '{s+=$2} END {print s}'

sucotronic · 2013-01-09 15:26:45 43
Don't like the cut command? Tired of typing awk '{print $xxx}', try this

Show Sample Output
This is sample output - yours may be different.
```
# ap () { awk '{print $'$1'}'; }
# echo hi there | ap 2
there
```
2

awp () { awk '{print $'$1'}'; }

joeheyming · 2014-08-15 16:21:49 9
drop first column of output by piping to this
This is sample output - yours may be different.
2

awk '{for(i=2;i<=NF;i++) printf("%s%s",$i,(i!=NF)?OFS:ORS)}'

snaguber · 2015-05-26 23:14:39 12
Monitor memory fine-grained usage (e.g. firefox)

Sometimes top/htop don't give the fine-grained detail on memory usage you might need. Sum up the exact memory types you want
This is sample output - yours may be different.
2

watch "awk '/Rss/{sum += \$2; } END{print sum, \"kB\"}' < /proc/$(pidof firefox)/smaps"

gumnos · 2015-09-19 00:36:34 18
Calculates the number of physical cores considering HyperThreading in AWK

Check whether hyperthreading is enabled or not. a better solution as nproc should work on all OS with awk Show Sample Output
This is sample output - yours may be different.
```
# example: 4xCPU 24xHT Cores
48
```
2

awk -F: '/^core id/ && !P[$2] { CORES++; P[$2]=1 }; /^physical id/ && !N[$2] { CPUs++; N[$2]=1 }; END { print CPUs*CORES }' /proc/cpuinfo

emphazer · 2018-05-14 14:17:33 158
Ultra fast public IP address lookup using Cloudflare's 1.1.1.1
This is sample output - yours may be different.
2

curl -fSs https://1.1.1.1/cdn-cgi/trace | awk -F= '/ip/ { print $2 }'

skhaz · 2019-09-05 19:31:25 246
Print nic name of current connection
This is sample output - yours may be different.
2

nmcli -g name,type connection show --active|awk -F: '/ethernet|wireless/ { print $1 }'

wuseman1 · 2022-09-22 20:16:21 662
urldecode with AWK

Fast and simple awk urldecoder! Note: Parameter -n is specific to GNU awk Show Sample Output
This is sample output - yours may be different.
```
echo -e  "asdf%2Fxx%2Cx\n%2F%2Cxx"  | awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..
asdf/xx,x
/,xx
```
2

awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..

emphazer · 2023-03-30 15:05:58 318
Display duplicated lines in a file

Displays the duplicated lines in a file and their occuring frequency.
This is sample output - yours may be different.
1

cat file.txt | sort | uniq -dc

Vadi · 2009-03-21 18:15:14 7
sync svn working copy and remote repository (auto adding new files)

Lists the local files that are not present in the remote repository (lines beginning with ?) and add them. Show Sample Output
This is sample output - yours may be different.
```
$ svn status | grep '^?'
?      INFO
?      src/svn-commit.tmp
?      src/game/Game.java
?      src/game/character/Hero.java

$ svn status | grep '^?' | awk '{ print $2; }' | xargs svn add
$ svn ci

A      INFO
A      src/svn-commit.tmp
A      src/game/Game.java
A      src/game/character/Hero.java
```
1

svn status | grep '^?' | awk '{ print $2; }' | xargs svn add

unixmonkey4200 · 2009-04-10 21:55:37 9
Uncompress a CSS file

Ever compress a file for the web by replacing all newline characters with nothing so it makes one nice big blob? It is a great idea, however what about when you want to edit that file? ...Serious pain in the butt. I ran into this today in that my only copy of a CSS file was "compressed" with no newlines. I whipped this up and it converted back into nice human readable CSS :-) It could be nicer, but it does the job.
This is sample output - yours may be different.
1

cat somefile.css | awk '{gsub(/{|}|;/,"&\n"); print}' >> uncompressed.css

lrvick · 2009-06-02 15:51:51 37
Puts every word from a file into a new line

Basically it creates a typical word list file from any normal text.
This is sample output - yours may be different.
1

awk '{c=split($0, s); for(n=1; n<=c; ++n) print s[n] }' INPUT_FILE > OUTPUT_FILE

agony · 2009-07-06 06:10:21 8
Remove newlines from output

?Cat and grep? You can use only grep ("grep \. filename"). Better option is awk.
This is sample output - yours may be different.
1

awk /./ filename

point_to_null · 2009-08-09 02:04:46 4
Sum columns from CSV column $COL

More of the same but with more elaborate perl-fu :-)
This is sample output - yours may be different.
1

perl -F',' -ane '$a += $F[3]; END { print $a }' test.csv

coffeeaddict_nl · 2009-08-11 15:08:58 3
Checks apache's access_log file, strips the search queries and shoves them up your e-mail

as unixmonkey7109 pointed out, first awk parse replaces three steps.
This is sample output - yours may be different.
1

awk '/q=/{print $11}' /var/log/httpd/access_log.4 | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1

isma · 2009-11-22 17:13:48 69
random xkcd comic as xml
This is sample output - yours may be different.
1

curl -sL 'dynamic.xkcd.com/comic/random/' | awk -F\" '/^<img/{printf("<?xml version=\"1.0\"?>\n<xkcd>\n<item>\n <title>%s</title>\n <comment>%s</comment>\n <image>%s</image>\n</item>\n</xkcd>\n", $6, $4, $2)}'

putnamhill · 2009-12-09 15:02:32 4
ShadyURL via CLI
This is sample output - yours may be different.
1

SITE="www.google.com"; curl --silent "http://www.shadyurl.com/create.php?myUrl=$SITE&shorten=on" | awk -F\' '/is now/{print $6}'

allrightname · 2010-02-19 14:21:02 6
Extract title from HTML files

previous version leaves lots of blank lines
This is sample output - yours may be different.
1

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' | sed '/^$/d' > file.html

tamouse · 2010-04-20 13:27:47 4
Extract title from HTML files

not the best, uses 4 pipes!
This is sample output - yours may be different.
1

tr -d "\n\r" | grep -ioEm1 "<title[^>]*>[^<]*</title" | cut -f2 -d\> | cut -f1 -d\<

bandie91 · 2010-04-20 18:55:24 3
Count TCP States From Netstat

Counts TCP states from Netstat and displays in an ordered list. Show Sample Output
This is sample output - yours may be different.
```
9 ESTABLISHED
9 LISTEN
1 SYN_SENT
```
1

netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c

Kered557 · 2010-05-06 17:04:37 4
Show DeviceMapper names for LVM Volumes (to disambiguate iostat logs, etc)

Emits the device names which will be printed by iostat for an LVM volume; doesn't show the names for the underlying devices when snapshots are being used (the -cow and -real devices in /dev/mapper) Show Sample Output
This is sample output - yours may be different.
```
dm-0 /dev/storage/root
dm-1 /dev/storage/swap_1
```
1

sudo lvdisplay |awk '/LV Name/{blockdev=$3} /Block device/{bdid=$3; sub("[0-9]*:","dm-",bdid); print bdid,blockdev;}'

unixmonkey10270 · 2010-06-10 11:50:31 3
Add all unversioned files to svn

No need for grep, let awk do the match. This will not behave properly if the filenames contains whitespace, which is awk's default field separator.
This is sample output - yours may be different.
1

svn st | awk '{if ($1 ~ "?") print $2}' | xargs svn add

sciurus · 2010-06-19 03:07:26 6
Edit a file in vim (at the first error) if it is not well formed xml.

Validate a file using xmllint. If there are parser errors, edit the file in vim at the line of the first error.
This is sample output - yours may be different.
1

vimlint(){ eval $(xmllint --noout "$1" 2>&1 | awk -F: '/parser error/{print "vim \""$1"\" +"$2; exit}'); }

putnamhill · 2010-06-23 15:55:02 3
‹ First < 3 4 5 6 7 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Benchmark SQL Query

Benchmark a SQL query against MySQL Server. The example runs the query 10 times, and you get the average runtime in the output. To ensure that the query does not get cached, use `RESET QUERY CACHE;` on top in the query file.

View Processeses like a fu, fu

Shows a less detailed output, made only of the process tree and their pids.

Which processes are listening on a specific port (e.g. port 80)

swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

List files with full path

This version is a bit more portable although it isn't extended as easily with '-type f' etc. On AIX the find command doesn't have -maxdepth or equivalent.

Delete a file/directory walking subdirectories (bash4 or zsh)

Find the 10 users that take up the most disk space

In OSX you would have to make sure that you "sudo -s" your way to happiness since it will give a few "Permission denied" errors before finally spitting out the results. In OSX the directory structure has to start with the "Users" Directory then it will recursively perform the operation. Your Lord and master, Mematron

capture mysql queries sent to server

Shows size of dirs and files, hidden or not, sorted.

Very useful when you need disk space. It calculates the disk usage of all files and dirs (descending them) located at the current directory (including hidden ones). Then sort puts them in order.

Decrypt passwords from Google Chrome and Chromium.

Read this before you down voting and comment that it is not working -> Wont work on latest versions ~75> since database file is locked and has to be decrypted. This is useful if you have an old hdd with a chrome installation and want to decrypt your old passwords fast.

Create md5sum of a directory

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Commands tagged awk (348) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged awk (348)

sorted by