Commands tagged awk | commandlinefu.com

Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

OpenID

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Hide

News

2011-03-12 - Confoo 2011 presentation: Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands: To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner: Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed: Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.

Hide

Functions

Hide

Credits

Site by David Winterbottom (user root).

Terminal - Commands tagged awk - 270 results

domain=google.com; for ns in $(whois $domain | awk -F: '/Name Server/{print $2}'); do echo ">>> Nameservers for $domain from $a <<<"; dig @$ns $domain ns +short; echo; done;

2011-05-08 04:46:34

User: laebshade

Functions: awk dig echo whois

Tags: awk grep for dig dns whois for loop nameserver glue record

Up
Down

Check to make sure the whois nameservers match the nameserver records from the nameservers themselves

Change the $domain variable to whichever domain you wish to query.

Works with the majority of whois info; for some that won't, you may have to compromise:

domain=google.com; for a in $(whois $domain | grep "Domain servers in listed order:" --after 3 | grep -v "Domain servers in listed order:"); do echo ">>> Nameservers for $domain from $a

Note that this doesn't work as well as the first one; if they have more than 3 nameservers, it won't hit them all.

As the summary states, this can be useful for making sure the whois nameservers for a domain match the nameserver records (NS records) from the nameservers themselves.

ls *.jpg | awk -F'.' '{ printf "%s %04d.%s\n", $0, $1, $2; }' | xargs -n2 mv

2011-05-01 13:32:58

User: hute37

Functions: awk ls printf xargs

Tags: awk mv xargs

Up
Down

Numeric zero padding file rename

rename file name with fixed length nomeric format pattern

find . -maxdepth 1 -type f -not -iname '*.jpg' -ls |awk '{TOTAL+=$7} END {print int(TOTAL/(1024^2))"MB"}'

2011-04-26 18:18:37

User: mack

Functions: awk find

Tags: awk find filesize

Up
Down

How to estimate the storage size of all files not named *.[extension] on the current directory

With this sentence we can estimate the storage size of all files not named *.jpg on the current directory.

The syntax is based on Linux, for Unix compliance use:

find ./* -prune ! -name '*.jpg' -ls |awk '{TOTAL+=$7} END {print int(TOTAL/(1024^2))"MB"}'

We can change the jpg extension for whatever extension what we need

sudo aptitude remove -P $(dpkg -l|awk '/^ii linux-image-2/{print $2}'|sed 's/linux-image-//'|awk -v v=`uname -r` 'v>$0'|sed 's/-generic//'|awk '{printf("linux-headers-%s\nlinux-headers-%s-generic\nlinux-image-%s-generic\n",$0,$0,$0)}')

2011-04-25 05:19:57

User: Bonster

Functions: awk sed sudo

Tags: Debian Linux awk grep kernel dpkg aptitude

-1

Up
Down

Remove all unused kernels with apt-get

Same as 7272 but that one was too dangerous

so i added -P to prompt users to continue or cancel

Note the double space: "...^ii␣␣linux-image-2..."

Like 5813, but fixes two bugs: [1]This leaves the meta-packages 'linux-headers-generic' and 'linux-image-generic' alone so that automatic upgrades work correctly in the future. [2]Kernels newer than the currently running one are left alone (this can happen if you didn't reboot after installing a new kernel).

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

VAR="foo" ; awk '{ print '"$VAR"' }'

2011-04-15 07:56:20

User: FRUiT

Functions: awk

Tags: bash awk print

-2

Up
Down

BASH: Print shell variable into AWK

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

MyVAR=86; awk -v n=$MyVAR '{print n}'

2011-04-14 17:56:00

User: putnamhill

Functions: awk

Tags: bash awk print

Up
Down

BASH: Print shell variable into AWK

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

MyVAR=85 awk '{ print ENVIRON["MyVAR"] }'

2011-04-14 16:46:23

User: depesz

Functions: awk

Tags: bash awk print

Up
Down

BASH: Print shell variable into AWK

Alternatively:

export MyVAR=84; awk '{ print ENVIRON["MyVAR"] }'

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

MyVAR=84; awk '{ print "'"$MyVAR"'" }'

2011-04-14 15:49:50

User: EBAH

Functions: awk

Tags: bash awk print

-3

Up
Down

BASH: Print shell variable into AWK

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

svn diff -r 1792:HEAD --summarize | awk '{if ($1 != "D") print $2}'| xargs -I {} tar rf incremental_release.tar {}

2011-04-05 15:00:49

User: windfold

Functions: awk diff tar xargs

Tags: bash svn awk xargs tar

Up
Down

Create a tar of modified/added files since revision 1792.

The result of this command is a tar with all files that have been modified/added since revision 1792 until HEAD. This command is super useful for incremental releases.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

ps -C apache o pid= | sed 's/^/-p /' | xargs strace

2011-03-15 08:46:33

User: depesz

Functions: ps sed xargs

Tags: awk token ps strace tokenizer

-1

Up
Down

easily strace all your apache processes

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

ps auxw | grep sbin/apache | awk '{print"-p " $2}' | xargs strace

This is sample output - yours may be different.

ps auxw | grep sbin/apache | awk '{print"-p " $2}' | xargs strace
Process 7313 attached - interrupt to quit
Process 7395 attached - interrupt to quit
Process 7396 attached - interrupt to quit
Process 7397 attached - interrupt to quit
Process 7398 attached - interrupt to quit
Process 7399 attached - interrupt to quit
Process 7401 attached - interrupt to quit
Process 7403 attached - interrupt to quit
Process 7448 attached - interrupt to quit
Process 7468 attached - interrupt to quit
Process 7469 attached - interrupt to quit
attach: ptrace(PTRACE_ATTACH, ...): No such process
[pid  7313] select(0, NULL, NULL, NULL, {0, 48000} <unfinished ...>
[pid  7395] poll([{fd=11, events=POLLIN}, {fd=10, events=POLLIN}, {fd=9, events=POLLIN}, {fd=8, events=POLLIN}, {fd=7, events=POLLIN}, {fd=6, events=POLLIN}, {fd=5, events=POLLIN}, {fd=4, events=POLLIN}, {fd=85, events=POLLIN}, {fd=19, events=POLLIN}], 10, -1 <unfinished ...>
[pid  7396] semop(1572864, {{0, -1, SEM_UNDO}}, 1 <unfinished ...>
[pid  7397] semop(1572864, {{0, -1, SEM_UNDO}}, 1 <unfinished ...>
[pid  7398] poll([{fd=15, events=POLLIN}, {fd=19, events=POLLIN}], 2, -1 <unfinished ...>
[pid  7399] semop(1572864, {{0, -1, SEM_UNDO}}, 1 <unfinished ...>
[pid  7401] poll([{fd=15, events=POLLIN}, {fd=19, events=POLLIN}], 2, -1 <unfinished ...>
[pid  7403] poll([{fd=15, events=POLLIN}, {fd=19, events=POLLIN}], 2, -1 <unfinished ...>
[pid  7448] poll([{fd=15, events=POLLIN}, {fd=19, events=POLLIN}], 2, -1 <unfinished ...>
[pid  7468] poll([{fd=15, events=POLLIN}, {fd=19, events=POLLIN}], 2, -1 <unfinished ...>
[pid  7469] poll([{fd=15, events=POLLIN}, {fd=19, events=POLLIN}], 2, -1 <unfinished ...>
[pid  7313] <... select resumed> )      = 0 (Timeout)
[pid  7313] gettimeofday({1300139401, 36665}, NULL) = 0
[pid  7313] waitpid(-1, 0xffdc0a68, WNOHANG|WSTOPPED) = 0
[pid  7313] select(0, NULL, NULL, NULL, {1, 0}) = 0 (Timeout)
[pid  7313] gettimeofday({1300139402, 39688}, NULL) = 0
[pid  7313] waitpid(-1, 0xffdc0a68, WNOHANG|WSTOPPED) = 0
[pid  7313] select(0, NULL, NULL, NULL, {1, 0} <unfinished ...>
Process 7313 detached
Process 7395 detached
Process 7396 detached
Process 7397 detached
Process 7398 detached
Process 7399 detached
Process 7401 detached
Process 7403 detached
Process 7448 detached
Process 7468 detached
Process 7469 detached

2011-03-14 21:45:22

User: px

Functions: awk grep ps xargs

Tags: awk token ps tokenizer

Up
Down

easily strace all your apache processes

This one-liner will use strace to attach to all of the currently running apache processes output and piped from the initial "ps auxw" command into some awk.

ls -1 | awk 'BEGIN{srand()} {x[NR] = $0} END{print "Selected", x[1 + int(rand() * NR)]}'

2011-03-13 20:05:06

User: saibbot

Functions: awk ls

Tags: awk random

-3

Up
Down

Selecting a random file/folder of a folder

I use this command to select a random movie from my movie collection..

ls -alt /directory/ | awk '{ print $6 " " $7 " -- " $9 }'

2011-02-17 12:50:38

User: gamefiend

Functions: awk ls

Tags: awk pipe ls

-3

Up
Down

Print last modified time in 'date -- file' format

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

lsof | awk '/*:https?/{print $2}' | sort -u

2011-02-04 01:37:17

User: sugitaro

Functions: awk sort

Tags: sort awk lsof

-1

Up
Down

pid list by httpd listen port

% lsof -v

lsof version information:

revision: 4.78

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk ' { printf ("%s ", $0)} END {printf ("\n") } ' FILE

2011-02-02 11:51:41

User: bouktin

Functions: awk printf

Tags: awk

-1

Up
Down

transform several lines in one with Awk

remove all carriage return of a given file (or input, if used with | ) and replace them with a space (or whatever character is after %s)

awk '{printf("/* %02d */ %s\n", NR,$0)}' inputfile > outputfile

2011-01-04 19:13:55

User: lucasrangit

Functions: awk

Tags: C Programming awk comment count

Up
Down

Add line number count as C-style comments

I often find the need to number enumerations and other lists when programming. With this command, create a new file called 'inputfile' with the text you want to number. Paste the contents of 'outputfile' back into your source file and fix the tabbing if necessary. You can also change this to output hex numbering by changing the "%02d" to "%02x". If you need to start at 0 replace "NR" with "NR-1". I adapted this from http://osxdaily.com/2010/05/20/easily-add-line-numbers-to-a-text-file/.

{ if (/^[A-Za-z0-9]/) { interface=$1; next } else { if (/inet [Aa][d]*r/) { split($2,ip,":") } else { next } } print interface"\t: "ip[2] }

2010-12-14 15:38:25

User: gismo82

Tags: ifconfig awk

Up
Down

Get your IP addresses

Shows the interface and the ip-address

awk 'BEGIN{RS="\0"}{gsub(/\n/,"<SOMETEXT>");print}' file.txt

2010-12-12 21:43:22

User: __

Functions: awk

Tags: Text Processing awk newline line

Up
Down

Join lines

awk version of 7210. Slightly longer, but expanding it to catch blank lines is easier:

awk 'BEGIN{RS="\0"}{gsub(/\n+/,"<SOMETEXT>");print}' file.txt

aptitude remove $(dpkg -l|awk '/^ii linux-image-2/{print $2}'|sed 's/linux-image-//'|awk -v v=`uname -r` 'v>$0'|sed 's/-generic//'|awk '{printf("linux-headers-%s\nlinux-headers-%s-generic\nlinux-image-%s-generic\n",$0,$0,$0)}')

2010-12-11 11:38:15

User: __

Functions: awk sed

Tags: Debian Linux awk grep kernel dpkg aptitude

Up
Down

Remove all unused kernels with apt-get

Note the double space: "...^ii␣␣linux-image-2..."

I'm bummed that this took 228 characters. I'd like to see a simpler version.

gawk '{n=$1;a=0;b=1;c=1;for(i=1;i<n;i++){c=a+b;a=b;b=c};print c}' << eof

2010-11-26 08:36:30

User: unixmonkey13930

Functions: gawk

Tags: awk

-5

Up
Down

Fibonacci numbers with awk

only take the first field on each row to compute the fibo on this number

tail -f file | awk '{now=strftime("%F %T%z\t");sub(/^/, now);print}'

2010-11-25 04:11:52

User: arcege

Functions: awk file tail

Tags: awk tail

Up
Down

Monitor a file with tail with timestamps added

Comments (2) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk 'func f(n){return(n<2?n:f(n-1)+f(n-2))}BEGIN{while(a<24){print f(a++)}}'

2010-11-24 10:40:08

User: unefunge

Functions: awk

Tags: awk

Up
Down

Fibonacci numbers with awk

recursive version, "pure" AWK

awk -F, '{print $1" "$2" "$NF}' foo.txt

2010-11-12 15:53:13

User: putnamhill

Functions: awk

Tags: awk field separator

Up
Down

AWK: Set Field Separator from command line

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk 'BEGIN {FS=","} { print $1 " " $2 " " $NF}' foo.txt

2010-11-12 15:26:04

User: EBAH

Functions: awk

Tags: awk field separator

-1

Up
Down

AWK: Set Field Separator from command line

Set field separator char from command line.

Prints first, second and lsat columns.

lynx -dump http://www.domain.com | awk '/http/{print $2}' | egrep "^https{0,1}"

2010-11-10 11:05:52

User: xutale

Functions: awk egrep

Tags: awk lynx

Up
Down

Get all links of a website

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

‹ First < 4 5 6 7 8 > Last ›