Commands tagged awk | commandlinefu.com

Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

OpenID

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

» all commands
» commands with 3 up-votes (commandlinefu3)
» commands with 10 up-votes (commandlinefu10)
» commands tagged awk

Hide

News

2011-03-12 - Confoo 2011 presentation: Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands: To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner: Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed: Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.

Hide

Functions

Hide

Credits

Site by David Winterbottom (user root).

Terminal - Commands tagged awk - 270 results

awk '(length>t) {t=length} END {print t}' *.cpp

2010-05-11 20:51:55

User: putnamhill

Functions: awk

Tags: awk

Up
Down

Length of longest line of code

Here's an awk version.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

tail -n2000 /var/www/domains/*/*/logs/access_log | awk '{print $1}' | sort | uniq -c | sort -n | awk '{ if ($1 > 20)print $1,$2}'

2010-05-10 19:08:37

User: allrightname

Functions: awk sort tail uniq

Tags: bash sort awk uniq tail IP apache

Up
Down

Count number of hits per IP address in last 2000 lines of apache logs and print the IP and hits if hits > 20

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

netstat -an | awk '/tcp/ {print $6}' | sort | uniq -c

2010-05-06 17:04:37

User: Kered557

Functions: awk netstat sort uniq

Tags: sort awk uniq netstat

Up
Down

Count TCP States From Netstat

Counts TCP states from Netstat and displays in an ordered list.

echo "Keyword?";read keyword;query="http://www.shoutcast.com/sbin/newxml.phtml?search="$keyword"";curl -s $query |awk -F '"' 'NR <= 4 {next}NR>15{exit}{sub(/SHOUTcast.com/,"http://yp.shoutcast.com/sbin/tunein-station.pls?id="$6)}{print i++" )"$2}'

2010-05-03 00:44:10

User: benyounes

Functions: awk echo

Tags: awk curl

Up
Down

Search shoutcast web radio by keyword

Searches for web radio by submitted keyword and returns the station name and the link for listing .

May be enhanced to read user's selection and submit it to mplayer.

history | awk '{a[$'$(echo "1 2 $HISTTIMEFORMAT" | wc -w)']++}END{for(i in a){print a[i] " " i}}' | sort -rn | head

2010-05-02 21:48:53

User: bandie91

Functions: awk echo sort wc

Tags: history awk wc

Up
Down

List of commands you use most often

If you use HISTTIMEFORMAT environment e.g. timestamping typed commands, $(echo "1 2 $HISTTIMEFORMAT" | wc -w)

gives the number of columns that containing non-command parts per lines.

It should universify this command.

tr -d "\n\r" | grep -ioEm1 "<title[^>]*>[^<]*</title" | cut -f2 -d\> | cut -f1 -d\<

2010-04-20 18:55:24

User: bandie91

Functions: cut grep tr

Tags: Linux awk html

Up
Down

Extract title from HTML files

not the best, uses 4 pipes!

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' | sed '/^$/d' > file.html

2010-04-20 13:27:47

User: tamouse

Functions: awk sed

Tags: Linux awk html

Up
Down

Extract title from HTML files

previous version leaves lots of blank lines

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

awk 'BEGIN{IGNORECASE=1;FS="<title>|</title>";RS=EOF} {print $2}' file.html

2010-04-20 10:54:03

User: sata

Functions: awk

Tags: Linux awk html

Up
Down

Extract title from HTML files

Case Insensitive! and Works even if the "<title>...</title>" spans over multiple line.

Simple! :-)

find ~/.mozilla/firefox/*/Cache -exec file {} \; | awk -F ': ' 'tolower($2)~/mpeg/{print $1}'

2010-04-19 06:59:55

User: sata

Functions: awk file find

Tags: awk find file mp3 firefox cache

Up
Down

Grab a list of MP3s out of Firefox's cache

Grab a list of MP3s (with full path) out of Firefox's cache

Ever gone to a site that has an MP3 embedded into a pesky flash player, but no download link? Well, this one-liner will yank the *full path* of those tunes straight out of FF's cache in a clean list.

Shorter and Intuitive version of the command submitted by (TuxOtaku)

ifconfig -a| awk '/^wlan|^eth|^lo/ {;a=$1;FS=":"; nextline=NR+1; next}{ if (NR==nextline) { split($2,b," ")}{ if ($2 ~ /[0-9]\./) {print a,b[1]}; FS=" "}}'

2010-04-15 04:34:28

User: alf

Functions: awk ifconfig

Tags: ifconfig awk IP

Up
Down

Show interface/ip using awk

Interfaces like lo can be omitted from the beginning, there are probably better ways of doing this, i'm a noob at awk.

for i in `ls ~/.mozilla/firefox/*/Cache`; do file $i | grep -i mpeg | awk '{print $1}' | sed s/.$//; done

2010-04-11 23:14:18

User: TuxOtaku

Functions: awk file grep sed

Tags: sed awk flash grep file mp3 firefox for mpeg cache

Up
Down

Grab a list of MP3s out of Firefox's cache

Ever gone to a site that has an MP3 embedded into a pesky flash player, but no download link? Well, this one-liner will yank the names of those tunes straight out of FF's cache in a nice, easy to read list. What you do with them after that is *ahem* no concern of mine. ;)

ffmpeg -f alsa -itsoffset 00:00:02.000 -ac 2 -i hw:0,0 -f x11grab -s $(xwininfo -root | grep 'geometry' | awk '{print $2;}') -r 10 -i :0.0 -sameq -f mp4 -s wvga -y intro.mp4

2010-03-31 09:33:05

User: mohan43u

Functions: awk grep

Tags: ffmpeg awk grep alsa xwininfo x11grab mp4 wvga

Up
Down

Capture screen and mic input using FFmpeg and ALSA

Yet another x11grab using ffmpeg. I also added mic input to the capturing video stream using alsa. Yet I need to find out how to capture audio which is currently playing.

ffmpeg -f x11grab -s `xdpyinfo | grep 'dimensions:'|awk '{print $2}'` -r 25 -i :0.0 -sameq /tmp/out.mpg > /root/howto/capture_screen_video_ffmpeg

2010-03-27 21:31:34

User: brubaker

Functions: awk grep

Tags: ffmpeg awk

Up
Down

Capture video of a linux desktop

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

alias busy='my_file=$(find /usr/include -type f | sort -R | head -n 1); my_len=$(wc -l $my_file | awk "{print $1}"); let "r = $RANDOM % $my_len" 2>/dev/null; vim +$r $my_file'

2010-03-09 21:48:41

User: busybee

Functions: alias awk find head sort vim wc

Tags: bash vim sort awk find wc

Up
Down

Makes you look busy

This makes an alias for a command named 'busy'. The 'busy' command opens a random file in /usr/include to a random line with vim. Drop this in your .bash_aliases and make sure that file is initialized in your .bashrc.

Comments (8) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

SITE="www.google.com"; curl --silent "http://www.shadyurl.com/create.php?myUrl=$SITE&shorten=on" | awk -F\' '/is now/{print $6}'

2010-02-19 14:21:02

User: allrightname

Functions: awk

Tags: bash awk grep curl ShadyURL

Up
Down

ShadyURL via CLI

Comments (3) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

function avg { awk "/$2/{sum += \$$1; lc += 1;} END {printf \"Average over %d lines: %f\n\", lc, sum/lc}"; }

2010-02-18 10:20:22

User: vimzard

Functions: awk

Tags: awk

Up
Down

Averaging columns of numbers

Computes a columns average in a file. Input parameters = column number and optional pattern.

curl -s http://www.commandlinefu.com/commands/by/$1/xml | awk -F'</?div[^>]*>' '/class=\"command\"/{gsub(/"/,"\"",$2); gsub(/</,"<",$2); gsub(/>/,">",$2); gsub(/&/,"\\&",$2); cmd=$2} /class=\"num-votes\"/{printf("%3i %s\n", $2, cmd)}'

2010-02-16 17:24:45

User: putnamhill

Functions: awk

Tags: awk curl commandlinefu points upvotes downvotes

Up
Down

Get your commandlinefu points (upvotes - downvotes)

This version prints current votes and commands for a user. Pass the user as an argument. While this technically "fits" as a one liner, it really is easier to look at as a shell script with extra whitespace. :)

xrandr -q | awk -F'current' -F',' 'NR==1 {gsub("( |current)","");print $2}'

2010-02-02 16:26:17

User: sputnick

Functions: awk

Tags: awk xrandr

Up
Down

Get current Xorg resolution via xrandr

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

fbemailscraper YourFBEmail Password

2010-01-31 00:44:35

User: dabom

Tags: bash awk curl facebook

Up
Down

Facebook Email Scraper

(Apparently it is too long so I put it in sample output, I hope that is OK.)

Run the long command (or put it in your .bashrc) in sample output then run:

fbemailscraper YourFBEmail Password

Voila! Your contacts' emails will appear.

Facebook seems to have gotten rid of the picture encoding of emails and replaced it with a text based version making it easy to scrape!

Needs curl to run and it was made pretty quickly so there might be bugs.

renice +5 -p $(pidof <process name>)

2010-01-19 22:16:24

User: 0x2142

Functions: pidof renice

Tags: awk grep nice awk grep renice

Up
Down

renice by name

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

file='path to file'; tar -cf - "$file" | pv -s $(du -sb "$file" | awk '{print $1}') | gzip -c | ssh -c blowfish user@host tar -zxf - -C /opt/games

2010-01-19 16:02:45

User: starchox

Functions: awk du file gzip ssh tar

Tags: ssh awk tar directory compress cipher

Up
Down

improve copy file over ssh showing progress

You set the file/dirname transfer variable, in the end point you set the path destination, this command uses pipe view to show progress, compress the file outut and takes account to change the ssh cipher. Support dirnames with spaces.

Merged ideas and comments by http://www.commandlinefu.com/commands/view/4379/copy-working-directory-and-compress-it-on-the-fly-while-showing-progress and http://www.commandlinefu.com/commands/view/3177/move-a-lot-of-files-over-ssh

lsof -p $(pidof firefox) | awk '/.mozilla/ { s = int($7/(2^20)); if(s>0) print (s)" MB -- "$9 | "sort -rn" }'

2010-01-13 22:45:53

User: tzk

Functions: awk pidof

Tags: sort awk firefox lsof

Up
Down

Check which files are opened by Firefox then sort by largest size.

Just refining last proposal for this check, showing awk power to make more complex math (instead /1024/1024, 2^20). We don't need declare variable before run lsof, because $(command) returns his output. Also, awk can perform filtering by regexp instead to call grep. I changed the 0.0000xxxx messy output, with a more readable form purging all fractional numbers and files less than 1 MB.

exipick -zi | xargs exim -Mrm

2010-01-13 21:53:43

User: markszabo

Functions: xargs

Tags: bash awk exim

-2

Up
Down

Purge frozen messages in Exim

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

for i in `mailq | awk '$6 ~ /^frozen$/ {print $3}'`; do exim -Mrm $i; done

2010-01-13 21:28:45

User: rjamestaylor

Functions: awk

Tags: bash awk exim

-2

Up
Down

Purge frozen messages in Exim

Although Exim will purge frozen (undeliverable) messages over time, the command "exim -Mrm #id#" where #id# is a particular message ID will purge a message immediately. Being lazy, I don't want to type the command for each frozen message, so I wrote the one-liner to do it for me.

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

$class=ExampleClass; $path=src; for constant in `grep ' const ' $class.php | awk '{print $2;}'`; do grep -r "$class::$constant" $path; done

2009-12-24 11:32:24

User: BluesDragon

Functions: awk grep

Tags: awk grep PHP

-2

Up
Down

Find all uses of PHP constants in a set of files

Comments (0) | Add to favourites | Report as malicious | Submit alternative | Report as a duplicate

‹ First < 6 7 8 9 10 > Last ›