Commands using awk | commandlinefu.com

Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

OpenID

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

UpGuard checks and validates configurations for every major OS, network device, and cloud provider.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands using awk

Hide

News

May 19, 2015 - A Look At The New Commandlinefu: I've put together a short writeup on what kind of newness you can expect from the next iteration of clfu. Check it out here.
March 2, 2015 - New Management: I'm Jon, I'll be maintaining and improving clfu. Thanks to David for building such a great resource!

Hide

Top Tags

awk, bash, curl, find, grep, Linux, sed

Hide

Functions

Terminal - Commands using awk - 1,218 results

awk '{print $1}' /var/log/httpd/access_log | sort | uniq -c | sort -rnk1 | head -n 10

2009-08-30 15:32:26

User: flip387

Functions: awk head sort uniq

Up
Down

Top 10 requestors by IP address from Apache/NCSA Logs

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

curl -s wap.kitco.com/exrate.wml | awk ' BEGIN { x=0; FS = "<" } { if ($0~"^<br/>") {x=0} if (x==1) {print $1} if ($0~"EUR/US") {x=1} }'

2009-08-27 16:30:19

User: jamaese

Functions: awk

Up
Down

Get Dollar-Euro exchage rate

You can get others rates changing the "EUR/US" part. look at the url: wap.kitco.com/exrate.wml to get more options.

cat /service/solr/log/main/current | tai64nlocal | grep "$`date '+%F %H'`\|`date '+%F %H %M' | awk '{print $1" 0"$2-1":"$3}'`$" | grep QTime | awk '{print $NF}' | awk -F\= '{ s += $2} END {print s/NR}'

2009-08-26 14:02:02

User: allrightname

Functions: awk cat grep

-2

Up
Down

Find the average QTime for all queries ran within the last hour for solr

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

echo 1234567890 | awk '{ print strftime("%c", $0); }'

2009-08-25 09:37:54

User: alvinx

Functions: awk echo

Up
Down

convert unixtime to human-readable with awk

- convert unixtime to human-readable with awk

- useful to read logfiles with unix-timestamps, f.e. squid-log:

sudo tail -f /var/log/squid3/access.log | awk '{ print strftime("%c ", $1) $0; }

echo `date +%m/%d/%y%X |awk '{print $1;}' `" => "` cat /proc/acpi/thermal_zone/THRM/temperature | awk '{print $2, $3;}'` >> datetmp.log

2009-08-24 21:26:29

User: ninadsp

Functions: awk cat echo

Up
Down

log your PC's motherboard and CPU temperature along with the current date

Uses the data in the /proc system, provided by the acpid, to find out the CPU temperature. Can be run on systems without lm-sensors installed as well.

find /dir | awk '{print length, $0}' | sort -nr | sed 's/^[[:digit:]]* //' | while read dirfile; do outfile="$(echo "$(basename "$dirfile")" | unaccent UTF-8)"; mv "$dirfile" "$(dirname "$dirfile")/$outfile"; done

2009-08-24 21:24:18

User: Patola

Functions: awk basename find mv read sed sort

Up
Down

Unaccent an entire directory tree with files.

This command changes all filename and directories within a directory tree to unaccented ones. I had to do this to 'sanitize' some samba-exported trees. The reason it works might seem a little difficult to see at first - it first reverses-sort by pathname length, then it renames only the basename of the path. This way it'll always go in the right order to rename everything.

Some notes:

1. You'll have to have the 'unaccent' command. On Ubuntu, just aptitude install unaccent.

2. In this case, the encoding of the tree was UTF-8 - but you might be using another one, just adjust the command to your encoding.

3. The program might spit a few harmless errors saying the files are the same - not to fear.

Comments (1) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk '!a[$0]++' file

2009-08-23 15:28:43

User: voyeg3r

Functions: awk

Up
Down

print file without duplicated lines using awk

This create an array 'a' with wole lines. only one occurrence of each line - Not Get lines ++ !

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk '!($0 in a) {a[$0];print}' file

2009-08-23 12:54:41

User: voyeg3r

Functions: awk

Up
Down

print file without duplicated lines usind awk

show file withou duplicated lines

Comments (1) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

for i in `seq 1 255`; do ping -c 1 10.10.10.$i | tr \\n ' ' | awk '/1 received/ {print $2}'; done

2009-08-22 09:34:27

User: hemanth

Functions: awk ping tr

Tags: ping

Up
Down

Ping sweep without NMAP

Comments (9) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

mailq | grep DrWEB | awk {'print $1'} | sed s/*//g | postsuper -d -

2009-08-21 22:01:41

User: foob4r

Functions: awk grep mailq sed

Tags: Postfix

Up
Down

delete all DrWeb status, failure and other messages on a postfix server

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

ps -ef | awk '/process-name/ && !/awk/ {print}'

2009-08-19 11:22:09

User: dopeman

Functions: awk ps

Up
Down

Search for an active process without catching the search-process

This does the same thing as many of the 'grep' based alternatives but allows a more finite control over the output. For example if you only wanted the process ID you could change the command:

ps -ef | awk '/mingetty/ && !/awk/ {print $2}'

If you wanted to kill the returned PID's:

ps -ef | awk '/mingetty/ && !/awk/ {print $2}' | xargs -i kill {}

mysql -u<user> -p<password> -s -e 'DESCRIBE <table>' <database> | tail -n +1 | awk '{ printf($1",")}' | head -c -1

2009-08-17 12:54:44

User: Cowboy

Functions: awk head tail

Up
Down

Get table column names from an MySQL-database in comma-seperated form

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk < file.name '{ system("resolveip -s " $1) }'

2009-08-17 08:09:39

User: alanwilliamson

Functions: awk

Up
Down

Resolve a list of domain names to IP addresses

Given a file of FQDN, this simple command resolves the IP addresses of those Useful for log files or anything else that outputs domain names.

FFPID=$(pidof firefox-bin) && lsof -p $FFPID | awk '{ if($7>0) print ($7/1024/1024)" MB -- "$9; }' | grep ".mozilla" | sort -rn

2009-08-16 08:58:22

User: josue

Functions: awk grep pidof sort

Tags: sort awk grep firefox lsof

Up
Down

Check which files are opened by Firefox then sort by largest size.

Check which files are opened by Firefox then sort by largest size (in MB). You can see all files opened by just replacing grep to "/". Useful if you'd like to debug and check which extensions or files are taking too much memory resources in Firefox.

egrep "^lease" /var/lib/dhcp/db/dhcpd.leases |awk '{ print $2 }'

2009-08-14 07:31:00

User: peshay

Functions: awk egrep

Up
Down

get IPs with a DHCP lease

You can use that to create a excludefile for nmap, to find hosts, with no DHCP lease in your DHCP range.

echo $PATH|awk -F: ' { for (i=1; i <= NF; i++) print $i }'

2009-08-12 07:56:56

User: nitins

Functions: awk echo

Tags: bash awk

-2

Up
Down

List path of binaries

Comments (1) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk -F ',' '{ x = x + $4 } END { print x }' test.csv

2009-08-11 12:10:33

User: coffeeaddict_nl

Functions: awk

Tags: awk column CSV sum

Up
Down

Sum columns from CSV column $COL

Comments (79) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

find . -name \*.pdf -exec pdfinfo {} \; | grep Pages | sed -e "s/Pages:\s*//g" | awk '{ sum += $1;} END { print sum; }'

2009-08-10 16:28:24

User: unixmonkey3987

Functions: awk find grep sed sum

Up
Down

Count the number of pages of all PDFs in current directory and all subdirs, recursively

Comments (2) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

ls -1 *.jpg | while read fn; do export pa=`exiv2 "$fn" | grep timestamp | awk '{ print $4 " " $5 ".jpg"}' | tr ":" "-"`; mv "$fn" "$pa"; done

2009-08-10 00:52:22

User: axanc

Functions: awk export grep ls mv read tr

Up
Down

Picture Renamer

Renames all the jpg files as their timestamps with ".jpg" extension.

awk /./ filename

2009-08-09 02:04:46

User: point_to_null

Functions: awk

Tags: awk

Up
Down

Remove newlines from output

?Cat and grep? You can use only grep ("grep \. filename"). Better option is awk.

Comments (1) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

nslookup commandlinefu.com|sed 's/[^0-9. ]//g'|tail -n 1|awk -F " " '{print $2}'

2009-08-07 17:32:55

User: thundernode

Functions: awk nslookup sed tail

Up
Down

Get the ip registered to a domain on OpenWRT

I use this in a script on my openwrt router to check if my DynDNS needs to be updated, saves your account from being banned for blank updates.

ifconfig | grep "0xffffffff" | awk '{ print $2 }' | xargs -n 1 ifconfig em0 delete

2009-08-04 05:18:36

User: vwal

Functions: awk grep ifconfig xargs

Tags: Network bsd FreeBSD aliases rc.conf

Up
Down

Delete all aliases for a network interface on a (Free)BSD system

The example command deletes all aliases for network interface 'em0' assuming that the aliases have netmask of 255.255.255.255 and the master IP has some other netmask (such as 255.255.255.0). See here -> http://my.galagzee.com/2009/07/22/deleting-all-network-interface-aliases/ for more on the rationale of this command.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

curl -s http://isc.sans.org/sources.html|grep "ipinfo.html"|awk -F"ip=" {'print $2'}|awk -F"\"" {'print $1'}|xargs -n1 sudo iptables -A OUTPUT -j DROP -d > 2&>1

2009-08-03 20:18:41

User: din7

Functions: awk grep iptables sudo xargs

Up
Down

Retrieve top ip threats from http://isc.sans.org/sources.html and add them into iptables output chain.

Comments (9) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

du -aB1m|awk '$1 >= 100'

2009-08-01 20:16:33

User: karatatar

Functions: awk du

Tags: Filesystem disk usage size

Up
Down

Which files/dirs waste my disk space

Useful for analyzing disk usage. If you prefer GUI try http://en.wikipedia.org/wiki/Filelight or http://www.marzocca.net/linux/baobab/

netstat -ant | grep :80 | grep ESTABLISHED | awk '{print $5}' | awk -F: '{print $1}' | sort | uniq -c | sort -n

2009-07-31 18:28:16

User: garchacesaug

Functions: awk grep netstat sort uniq

Up
Down

Show in a web server, running in the port 80, how many ESTABLISHED connections by ip it has.

The command could show you all conecctions if you skip "grep ESTABLISHED"

‹ First < 39 40 41 42 43 > Last ›