Commands tagged awk sorted by votes

Commands tagged awk (348)

sorted by

Convert number of bytes to human readable filesize

Converts a number of bytes provided as input, to a human readable number. Show Sample Output
This is sample output - yours may be different.
```
$ human_filesize 1234567890
1.15 Gb
```
2

human_filesize() { awk -v sum="$1" ' BEGIN {hum[1024^3]="Gb"; hum[1024^2]="Mb"; hum[1024]="Kb"; for (x=1024^3; x>=1024; x/=1024) { if (sum>=x) { printf "%.2f %s\n",sum/x,hum[x]; break; } } if (sum<1024) print "1kb"; } '}

ArtBIT · 2011-12-02 18:21:20 3
Get the date for the last Saturday of a given month

If your locale has Monday as the first day of the week, like mine in the UK, change the two $7 into $6 Show Sample Output
This is sample output - yours may be different.
```
28
```
2

cal 04 2012 | awk '{ $7 && X=$7 } END { print X }'

flatcap · 2012-05-06 23:43:21 40
Count the number of pages of all PDFs in current directory and all subdirs, recursively
This is sample output - yours may be different.
2

find . -name "*.pdf" -exec pdftk {} dump_data output \; | grep NumberOfPages | awk '{s+=$2} END {print s}'

sucotronic · 2013-01-09 15:26:45 43
Don't like the cut command? Tired of typing awk '{print $xxx}', try this

Show Sample Output
This is sample output - yours may be different.
```
# ap () { awk '{print $'$1'}'; }
# echo hi there | ap 2
there
```
2

awp () { awk '{print $'$1'}'; }

joeheyming · 2014-08-15 16:21:49 9
drop first column of output by piping to this
This is sample output - yours may be different.
2

awk '{for(i=2;i<=NF;i++) printf("%s%s",$i,(i!=NF)?OFS:ORS)}'

snaguber · 2015-05-26 23:14:39 12
Monitor memory fine-grained usage (e.g. firefox)

Sometimes top/htop don't give the fine-grained detail on memory usage you might need. Sum up the exact memory types you want
This is sample output - yours may be different.
2

watch "awk '/Rss/{sum += \$2; } END{print sum, \"kB\"}' < /proc/$(pidof firefox)/smaps"

gumnos · 2015-09-19 00:36:34 18
Calculates the number of physical cores considering HyperThreading in AWK

Check whether hyperthreading is enabled or not. a better solution as nproc should work on all OS with awk Show Sample Output
This is sample output - yours may be different.
```
# example: 4xCPU 24xHT Cores
48
```
2

awk -F: '/^core id/ && !P[$2] { CORES++; P[$2]=1 }; /^physical id/ && !N[$2] { CPUs++; N[$2]=1 }; END { print CPUs*CORES }' /proc/cpuinfo

emphazer · 2018-05-14 14:17:33 160
Ultra fast public IP address lookup using Cloudflare's 1.1.1.1
This is sample output - yours may be different.
2

curl -fSs https://1.1.1.1/cdn-cgi/trace | awk -F= '/ip/ { print $2 }'

skhaz · 2019-09-05 19:31:25 247
Print nic name of current connection
This is sample output - yours may be different.
2

nmcli -g name,type connection show --active|awk -F: '/ethernet|wireless/ { print $1 }'

wuseman1 · 2022-09-22 20:16:21 662
urldecode with AWK

Fast and simple awk urldecoder! Note: Parameter -n is specific to GNU awk Show Sample Output
This is sample output - yours may be different.
```
echo -e  "asdf%2Fxx%2Cx\n%2F%2Cxx"  | awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..
asdf/xx,x
/,xx
```
2

awk -niord '{printf RT?$0chr("0x"substr(RT,2)):$0}' RS=%..

emphazer · 2023-03-30 15:05:58 319
Display duplicated lines in a file

Displays the duplicated lines in a file and their occuring frequency.
This is sample output - yours may be different.
1

cat file.txt | sort | uniq -dc

Vadi · 2009-03-21 18:15:14 7
sync svn working copy and remote repository (auto adding new files)

Lists the local files that are not present in the remote repository (lines beginning with ?) and add them. Show Sample Output
This is sample output - yours may be different.
```
$ svn status | grep '^?'
?      INFO
?      src/svn-commit.tmp
?      src/game/Game.java
?      src/game/character/Hero.java

$ svn status | grep '^?' | awk '{ print $2; }' | xargs svn add
$ svn ci

A      INFO
A      src/svn-commit.tmp
A      src/game/Game.java
A      src/game/character/Hero.java
```
1

svn status | grep '^?' | awk '{ print $2; }' | xargs svn add

unixmonkey4200 · 2009-04-10 21:55:37 9
Uncompress a CSS file

Ever compress a file for the web by replacing all newline characters with nothing so it makes one nice big blob? It is a great idea, however what about when you want to edit that file? ...Serious pain in the butt. I ran into this today in that my only copy of a CSS file was "compressed" with no newlines. I whipped this up and it converted back into nice human readable CSS :-) It could be nicer, but it does the job.
This is sample output - yours may be different.
1

cat somefile.css | awk '{gsub(/{|}|;/,"&\n"); print}' >> uncompressed.css

lrvick · 2009-06-02 15:51:51 37
Puts every word from a file into a new line

Basically it creates a typical word list file from any normal text.
This is sample output - yours may be different.
1

awk '{c=split($0, s); for(n=1; n<=c; ++n) print s[n] }' INPUT_FILE > OUTPUT_FILE

agony · 2009-07-06 06:10:21 8
Remove newlines from output

?Cat and grep? You can use only grep ("grep \. filename"). Better option is awk.
This is sample output - yours may be different.
1

awk /./ filename

point_to_null · 2009-08-09 02:04:46 4
Sum columns from CSV column $COL

More of the same but with more elaborate perl-fu :-)
This is sample output - yours may be different.
1

perl -F',' -ane '$a += $F[3]; END { print $a }' test.csv

coffeeaddict_nl · 2009-08-11 15:08:58 3
Checks apache's access_log file, strips the search queries and shoves them up your e-mail

as unixmonkey7109 pointed out, first awk parse replaces three steps.
This is sample output - yours may be different.
1

awk '/q=/{print $11}' /var/log/httpd/access_log.4 | awk -F 'q=' '{print $2}' | sed 's/+/ /g;s/%22/"/g;s/q=//' | cut -d "&" -f 1

isma · 2009-11-22 17:13:48 69
random xkcd comic as xml
This is sample output - yours may be different.
1

curl -sL 'dynamic.xkcd.com/comic/random/' | awk -F\" '/^<img/{printf("<?xml version=\"1.0\"?>\n<xkcd>\n<item>\n <title>%s</title>\n <comment>%s</comment>\n <image>%s</image>\n</item>\n</xkcd>\n", $6, $4, $2)}'

putnamhill · 2009-12-09 15:02:32 4
ShadyURL via CLI
This is sample output - yours may be different.
1

SITE="www.google.com"; curl --silent "http://www.shadyurl.com/create.php?myUrl=$SITE&shorten=on" | awk -F\' '/is now/{print $6}'

allrightname · 2010-02-19 14:21:02 6
Extract title from HTML files

previous version leaves lots of blank lines
This is sample output - yours may be different.
1

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' | sed '/^$/d' > file.html

tamouse · 2010-04-20 13:27:47 4
Extract title from HTML files

not the best, uses 4 pipes!
This is sample output - yours may be different.
1

tr -d "\n\r" | grep -ioEm1 "<title[^>]*>[^<]*</title" | cut -f2 -d\> | cut -f1 -d\<

bandie91 · 2010-04-20 18:55:24 3
Count TCP States From Netstat

Counts TCP states from Netstat and displays in an ordered list. Show Sample Output
This is sample output - yours may be different.
```
9 ESTABLISHED
9 LISTEN
1 SYN_SENT
```
1

netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c

Kered557 · 2010-05-06 17:04:37 4
Show DeviceMapper names for LVM Volumes (to disambiguate iostat logs, etc)

Emits the device names which will be printed by iostat for an LVM volume; doesn't show the names for the underlying devices when snapshots are being used (the -cow and -real devices in /dev/mapper) Show Sample Output
This is sample output - yours may be different.
```
dm-0 /dev/storage/root
dm-1 /dev/storage/swap_1
```
1

sudo lvdisplay |awk '/LV Name/{blockdev=$3} /Block device/{bdid=$3; sub("[0-9]*:","dm-",bdid); print bdid,blockdev;}'

unixmonkey10270 · 2010-06-10 11:50:31 3
Add all unversioned files to svn

No need for grep, let awk do the match. This will not behave properly if the filenames contains whitespace, which is awk's default field separator.
This is sample output - yours may be different.
1

svn st | awk '{if ($1 ~ "?") print $2}' | xargs svn add

sciurus · 2010-06-19 03:07:26 6
Edit a file in vim (at the first error) if it is not well formed xml.

Validate a file using xmllint. If there are parser errors, edit the file in vim at the line of the first error.
This is sample output - yours may be different.
1

vimlint(){ eval $(xmllint --noout "$1" 2>&1 | awk -F: '/parser error/{print "vim \""$1"\" +"$2; exit}'); }

putnamhill · 2010-06-23 15:55:02 3
‹ First < 3 4 5 6 7 > Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Get MD5 checksum from a pipe stream and do not alter it

last.fm rss parser

you can just use one awk script to parse the rss feed. No need to pipe so many awk's and sed's. Its ugly and inefficient.

Run a command, redirecting output to a file, then edit the file with vim.

This is one of those 'nothing' shell functions ...which I use all the time. If the command contains spaces, it must be quoted, e.g. $ vimcmd 'svn diff' /tmp/svndiff.out If I want to keep the output of the command that I'm running, I use vimcmd. If I don't need to keep the output, I use this: $ vim

Print current running shell, PID

works as well as echo $0, but also prints process id, which pts you're using. echo $SHELL doesn't always get updated when changing shells, so this is a better solution than that. Just one more variation on a theme.

Which processes are listening on a specific port (e.g. port 80)

swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Dump dvd from a different machine onto this one.

You can use this to directly dump from machine A (with dvd drive) to machine B (without dvd drive) . I used this to copy dvd using my friend's machine to my netbook. Above command is to be issued on machine B. Advantages : 1) No wasting time dumping first to machine A and then copying to Machine B. 2) You dont need to use space on Machine A. In fact, this will work even when Machine A doesnt have enough hdd space to dump the DVD. Use -C ssh option on slow networks (enables compression). you can replace "dd if=/dev/dvd" with any ripping command as long as it spews the iso to stdout.

grep tab (\t)

works in bash

Find the package that installed a command

Find usb device in realtime

Using this command you can track a moment when usb device was attached.

Convert seconds to [DD:][HH:]MM:SS

Converts any number of seconds into days, hours, minutes and seconds. sec2dhms() { declare -i SS="$1" D=$(( SS / 86400 )) H=$(( SS % 86400 / 3600 )) M=$(( SS % 3600 / 60 )) S=$(( SS % 60 )) [ "$D" -gt 0 ] && echo -n "${D}:" [ "$H" -gt 0 ] && printf "%02g:" "$H" printf "%02g:%02g\n" "$M" "$S" }

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Commands tagged awk (348) the last day the last week the last month all time sorted by date votes

What's this?

Check These Out

Stay in the loop…

Commands tagged awk (348)

sorted by