Commands tagged awk sorted by votes

Commands tagged awk (348)

sorted by

Convert number of bytes to human readable filesize

Converts a number of bytes provided as input, to a human readable number. Show Sample Output
This is sample output - yours may be different.
```
$ human_filesize 1234567890
1.15 Gb
```
2

human_filesize() { awk -v sum="$1" ' BEGIN {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; } '}

ArtBIT · 2011-12-02 18:21:20 3
Get the date for the last Saturday of a given month

If your locale has Monday as the first day of the week, like mine in the UK, change the two $7 into $6 Show Sample Output
This is sample output - yours may be different.
```
28
```
2

cal 04 2012 | awk '{ $7 && X=$7 } END { print X }'

flatcap · 2012-05-06 23:43:21 40
Count the number of pages of all PDFs in current directory and all subdirs, recursively
This is sample output - yours may be different.
2

find . -name "*.pdf" -exec pdftk {} dump_data output \; | grep NumberOfPages | awk '{s+=$2} END {print s}'

sucotronic · 2013-01-09 15:26:45 43
Don't like the cut command? Tired of typing awk '{print $xxx}', try this

Show Sample Output
This is sample output - yours may be different.
```
# ap () { awk '{print $'$1'}'; }
# echo hi there | ap 2
there
```
2

awp () { awk '{print $'$1'}'; }

joeheyming · 2014-08-15 16:21:49 9
drop first column of output by piping to this
This is sample output - yours may be different.
2

awk '{for(i=2;i<=NF;i++) printf("%s%s",$i,(i!=NF)?OFS:ORS)}'

snaguber · 2015-05-26 23:14:39 12
Monitor memory fine-grained usage (e.g. firefox)

Sometimes top/htop don't give the fine-grained detail on memory usage you might need. Sum up the exact memory types you want
This is sample output - yours may be different.
2

watch "awk '/Rss/{sum += \$2; } END{print sum, \"kB\"}' < /proc/$(pidof firefox)/smaps"

gumnos · 2015-09-19 00:36:34 18
Calculates the number of physical cores considering HyperThreading in AWK

Check whether hyperthreading is enabled or not. a better solution as nproc should work on all OS with awk Show Sample Output
This is sample output - yours may be different.
```
# example: 4xCPU 24xHT Cores
48
```
2

awk -F: '/^core id/ && !P[$2] { CORES++; P[$2]=1 }; /^physical id/ && !N[$2] { CPUs++; N[$2]=1 }; END { print CPUs*CORES }' /proc/cpuinfo

emphazer · 2018-05-14 14:17:33 160
Ultra fast public IP address lookup using Cloudflare's 1.1.1.1
This is sample output - yours may be different.
2

curl -fSs https://1.1.1.1/cdn-cgi/trace | awk -F= '/ip/ { print $2 }'

skhaz · 2019-09-05 19:31:25 247
Print nic name of current connection
This is sample output - yours may be different.
2

nmcli -g name,type connection show --active|awk -F: '/ethernet|wireless/ { print $1 }'

wuseman1 · 2022-09-22 20:16:21 662
urldecode with AWK

Fast and simple awk urldecoder! Note: Parameter -n is specific to GNU awk Show Sample Output
This is sample output - yours may be different.
```
echo -e  "asdf%2Fxx%2Cx\n%2F%2Cxx"  | awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..
asdf/xx,x
/,xx
```
2

awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..

emphazer · 2023-03-30 15:05:58 319
Display duplicated lines in a file

Displays the duplicated lines in a file and their occuring frequency.
This is sample output - yours may be different.
1

cat file.txt | sort | uniq -dc

Vadi · 2009-03-21 18:15:14 7
sync svn working copy and remote repository (auto adding new files)

Lists the local files that are not present in the remote repository (lines beginning with ?) and add them. Show Sample Output
This is sample output - yours may be different.
```
$ svn status | grep '^?'
?      INFO
?      src/svn-commit.tmp
?      src/game/Game.java
?      src/game/character/Hero.java

$ svn status | grep '^?' | awk '{ print $2; }' | xargs svn add
$ svn ci

A      INFO
A      src/svn-commit.tmp
A      src/game/Game.java
A      src/game/character/Hero.java
```
1

svn status | grep '^?' | awk '{ print $2; }' | xargs svn add

unixmonkey4200 · 2009-04-10 21:55:37 9
Uncompress a CSS file

Ever compress a file for the web by replacing all newline characters with nothing so it makes one nice big blob? It is a great idea, however what about when you want to edit that file? ...Serious pain in the butt. I ran into this today in that my only copy of a CSS file was "compressed" with no newlines. I whipped this up and it converted back into nice human readable CSS :-) It could be nicer, but it does the job.
This is sample output - yours may be different.
1

cat somefile.css | awk '{gsub(/{|}|;/,"&\n"); print}' >> uncompressed.css

lrvick · 2009-06-02 15:51:51 37
Puts every word from a file into a new line

Basically it creates a typical word list file from any normal text.
This is sample output - yours may be different.
1

awk '{c=split($0, s); for(n=1; n<=c; ++n) print s[n] }' INPUT_FILE > OUTPUT_FILE

agony · 2009-07-06 06:10:21 8
Remove newlines from output

?Cat and grep? You can use only grep ("grep \. filename"). Better option is awk.
This is sample output - yours may be different.
1

awk /./ filename

point_to_null · 2009-08-09 02:04:46 4
Sum columns from CSV column $COL

More of the same but with more elaborate perl-fu :-)
This is sample output - yours may be different.
1

perl -F',' -ane '$a += $F[3]; END { print $a }' test.csv

coffeeaddict_nl · 2009-08-11 15:08:58 3
Checks apache's access_log file, strips the search queries and shoves them up your e-mail

as unixmonkey7109 pointed out, first awk parse replaces three steps.
This is sample output - yours may be different.
1

awk '/q=/{print $11}' /var/log/httpd/access_log.4 | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1

isma · 2009-11-22 17:13:48 69
random xkcd comic as xml
This is sample output - yours may be different.
1

curl -sL 'dynamic.xkcd.com/comic/random/' | awk -F\" '/^<img/{printf("<?xml version=\"1.0\"?>\n<xkcd>\n<item>\n <title>%s</title>\n <comment>%s</comment>\n <image>%s</image>\n</item>\n</xkcd>\n", $6, $4, $2)}'

putnamhill · 2009-12-09 15:02:32 4
ShadyURL via CLI
This is sample output - yours may be different.
1

SITE="www.google.com"; curl --silent "http://www.shadyurl.com/create.php?myUrl=$SITE&shorten=on" | awk -F\' '/is now/{print $6}'

allrightname · 2010-02-19 14:21:02 6
Extract title from HTML files

previous version leaves lots of blank lines
This is sample output - yours may be different.
1

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' | sed '/^$/d' > file.html

tamouse · 2010-04-20 13:27:47 4
Extract title from HTML files

not the best, uses 4 pipes!
This is sample output - yours may be different.
1

tr -d "\n\r" | grep -ioEm1 "<title[^>]*>[^<]*</title" | cut -f2 -d\> | cut -f1 -d\<

bandie91 · 2010-04-20 18:55:24 3
Count TCP States From Netstat

Counts TCP states from Netstat and displays in an ordered list. Show Sample Output
This is sample output - yours may be different.
```
9 ESTABLISHED
9 LISTEN
1 SYN_SENT
```
1

netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c

Kered557 · 2010-05-06 17:04:37 4
Show DeviceMapper names for LVM Volumes (to disambiguate iostat logs, etc)

Emits the device names which will be printed by iostat for an LVM volume; doesn't show the names for the underlying devices when snapshots are being used (the -cow and -real devices in /dev/mapper) Show Sample Output
This is sample output - yours may be different.
```
dm-0 /dev/storage/root
dm-1 /dev/storage/swap_1
```
1

sudo lvdisplay |awk '/LV Name/{blockdev=$3} /Block device/{bdid=$3; sub("[0-9]*:","dm-",bdid); print bdid,blockdev;}'

unixmonkey10270 · 2010-06-10 11:50:31 3
Add all unversioned files to svn

No need for grep, let awk do the match. This will not behave properly if the filenames contains whitespace, which is awk's default field separator.
This is sample output - yours may be different.
1

svn st | awk '{if ($1 ~ "?") print $2}' | xargs svn add

sciurus · 2010-06-19 03:07:26 6
Edit a file in vim (at the first error) if it is not well formed xml.

Validate a file using xmllint. If there are parser errors, edit the file in vim at the line of the first error.
This is sample output - yours may be different.
1

vimlint(){ eval $(xmllint --noout "$1" 2>&1 | awk -F: '/parser error/{print "vim \""$1"\" +"$2; exit}'); }

putnamhill · 2010-06-23 15:55:02 3
‹ First < 3 4 5 6 7 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

start a vnc server session to connect to a gdm login screen

Mount a Windows share on the local network (Ubuntu) with user rights and use a specific samba user

find out which directories in /home have the most files currently open

a find and replace within text-based files

using find's exec option instead of a for loop and using sed's -i option for inplace replacement. no need to do the file swap.

Create a mirror of a local folder, on a remote server

Create a exact mirror of the local folder "/root/files", on remote server 'remote_server' using SSH command (listening on port 22) (all files & folders on destination server/folder will be deleted)

list block devices

Shows all block devices in a tree with descruptions of what they are.

Outputs files with ascii art in the intended form.

Files containing ascii art (e.g. with .nfo extension) are typically not correctly reproduced at the command line when using cat. With iconv one can easily write a wrapper to solve this: $ #!/bin/bash $ if [ -z "$@" ]; then echo "Usage: $(basename $0) file [file] ..." $ else iconv -f437 -tutf8 "$@"; fi $ exit 0

test for ksh/bash

Dave Korn gave me this one. It works because ksh allows variable names ( w/o the $name syntax ) used by sh and bash. I wrote it to permit "single source" shell libraries; the current objective: every shell library may be sourced by either shell. see http://github.com/applemcg/backash

Download all Phrack .tar.gzs

underscore to camelCase

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Commands tagged awk (348) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged awk (348)

sorted by