Commands tagged awk (313)

  • This bash function uses albumart.org to find the cover for an album. It returns an amazon.com url to the image. Usage: albumart [artist] [album] These arguments can be reversed and if the album name is distinct enough, it may be possible to omit the artist. The command can be extended with wget to automatically download the matching image like this: albumart(){ local x y="$@";x=$(awk '/View larger image/{gsub(/^.*largeImagePopup\(.|., .*$/,"");print;exit}' <(curl -s 'http://www.albumart.org/index.php?srchkey='${y// /+}'&itempage=1&newsearch=1&searchindex=Music'));[ -z "$x" ]&&echo "Not found."||wget "$x" -O "${y}.${x##*.}";} Show Sample Output


    7
    albumart(){ local y="$@";awk '/View larger image/{gsub(/^.*largeImagePopup\(.|., .*$/,"");print;exit}' <(curl -s 'http://www.albumart.org/index.php?srchkey='${y// /+}'&itempage=1&newsearch=1&searchindex=Music');}
    eightmillion · 2009-11-15 19:54:16 1
  • This function uploads images to http://omploader.org and then prints out the links to the file. Some coloring can also be added to the command with: ompload() { curl -F file1=@"$1" http://omploader.org/upload|awk '/Info:|File:|Thumbnail:|BBCode:/{gsub(/<[^<]*?\/?>/,"");$1=$1;sub(/^/,"\033[0;34m");sub(/:/,"\033[0m:");print}';} Show Sample Output


    8
    ompload() { curl -# -F file1=@"$1" http://ompldr.org/upload|awk '/Info:|File:|Thumbnail:|BBCode:/{gsub(/<[^<]*?\/?>/,"");$1=$1;print}';}
    eightmillion · 2009-11-07 20:56:52 1
  • Simple MAC adrress, thanks to ifconfig.


    0
    ifconfig | awk '/HW/ {print $5}'
    Cont3mpo · 2009-11-05 18:00:50 1

  • 4
    svn ci `svn stat |awk '/^A/{printf $2" "}'`
    realist · 2009-11-04 03:30:07 0
  • Sometimes, you don't really care about all the other information that ifconfig spits at you (however useful it may otherwise be). You just want an IP. This strips out all the crap and gives you exactly what you want. Show Sample Output


    2
    ifconfig eth1 | grep inet\ addr | awk '{print $2}' | cut -d: -f2 | sed s/^/eth1:\ /g
    TuxOtaku · 2009-11-03 19:26:40 3
  • Calculating series with awk only, no need for seq: add numbers from 1 to 100 Variations: 1+3+...+(2n-1) = n^2 awk 'BEGIN {for(i=1;i<=19;i+=2)sum+=i}; END {print sum}' /dev/null # displays 100 1/2 + 1/4 + ... = 1 awk 'BEGIN {for(i=1;i<=10;i++)sum+=1/(2**i)}; END {print sum}' /dev/null # displays 0.999023 Show Sample Output


    0
    awk 'BEGIN {for(i=1;i<=100;i++)sum+=i}; END {print sum}' /dev/null
    dennisw · 2009-10-26 18:24:57 0
  • Handled all within awk. Takes the value from $PWD and constructs directory structures and runs commands against them. The gsub() call is not necessary, but added for better visibility. If a variable DIR is given on the awk command-line, then that directory is used instead: awk -vDIR=$HOME/.ssh 'BEGIN{dir=DIR?...}' Show Sample Output


    -1
    awk 'BEGIN{dir=DIR?DIR:ENVIRON["PWD"];l=split(dir,parts,"/");last="";for(i=1;i<l+1;i++){d=last"/"parts[i];gsub("//","/",d);system("ls -ld \""d"\"");last=d}}'
    arcege · 2009-10-22 16:28:07 2

  • 6
    (echo "set terminal png;plot '-' u 1:2 t 'cpu' w linespoints;"; sudo vmstat 2 10 | awk 'NR > 2 {print NR, $13}') | gnuplot > plot.png
    syssyphus · 2009-09-23 16:40:13 6
  • draw `vmstat ` data using gnuplot


    0
    vmstat 2 10 | awk 'NR > 2 {print NR, $13}' | gnuplot -e "set terminal png;set output 'v.png';plot '-' u 1:2 t 'cpu' w linespoints;"
    Sunng · 2009-09-23 07:19:47 0
  • This pipeline will find, sort and display all files based on mtime. This could be done with find | xargs, but the find | xargs pipeline will not produce correct results if the results of find are greater than xargs command line buffer. If the xargs buffer fills, xargs processes the find results in more than one batch which is not compatible with sorting. Note the "-print0" on find and "-0" switch for perl. This is the equivalent of using xargs. Don't you love perl? Note that this pipeline can be easily modified to any data produced by perl's stat operator. eg, you could sort on size, hard links, creation time, etc. Look at stat and just change the '9' to what you want. Changing the '9' to a '7' for example will sort by file size. A '3' sorts by number of links.... Use head and tail at the end of the pipeline to get oldest files or most recent. Use awk or perl -wnla for further processing. Since there is a tab between the two fields, it is very easy to process. Show Sample Output


    3
    find $HOME -type f -print0 | perl -0 -wn -e '@f=<>; foreach $file (@f){ (@el)=(stat($file)); push @el, $file; push @files,[ @el ];} @o=sort{$a->[9]<=>$b->[9]} @files; for $i (0..$#o){print scalar localtime($o[$i][9]), "\t$o[$i][-1]\n";}'|tail
    drewk · 2009-09-21 22:11:16 4
  • I have a directory containing log files. This command delete all but the 5 latest logs. Here is how it works: * The ls -t command list all files with the latest ones at the top * The awk's expression means: for those lines greater than 5, delete.


    -2
    ls -t | awk 'NR>5 {system("rm \"" $0 "\"")}'
    haivu · 2009-09-16 04:58:08 0
  • This will calculate a running standard deviation in one pass and should never have the possibility for overflow that can happen with other implementations. I suppose there is a potential for underflow in the corner case where the deltas are small or the values themselves are small.


    4
    awk '{delta = $1 - avg; avg += delta / NR; mean2 += delta * ($1 - avg); } END { print sqrt(mean2 / NR); }'
    ashawley · 2009-09-11 04:46:01 2
  • This command displays a list of lines that are longer than 72 characters. I use this command to identify those lines in my scripts and cut them short the way I like it.


    16
    awk 'length>72' file
    haivu · 2009-09-10 05:54:41 3
  • Removes trailing newline; colon becomes record separator and newline becomes field separator, only the first field is ever printed. Replaces empty entries with $PWD. Also prepend relative directories (like ".") with the current directory ($PWD). Can change PWD with env(1) to get tricky in (non-Bourne) scripts. Show Sample Output


    -2
    echo src::${PATH} | awk 'BEGIN{pwd=ENVIRON["PWD"];RS=":";FS="\n"}!$1{$1=pwd}$1!~/^\//{$1=pwd"/"$1}{print $1}'
    arcege · 2009-09-09 04:03:46 0
  • This is really fast :) time find . -name \*.c | xargs wc -l | tail -1 | awk '{print $1}' 204753 real 0m0.191s user 0m0.068s sys 0m0.116s Show Sample Output


    0
    find . -name \*.c | xargs wc -l | tail -1 | awk '{print $1}'
    karpoke · 2009-09-08 08:25:45 1
  • Checks the Gmail ATOM feed for your account, parses it and outputs a list of unread messages. For some reason sed gets stuck on OS X, so here's a Perl version for the Mac: curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | perl -pe 's/^<title>(.*)<\/title>.*<name>(.*)<\/name>.*$/$2 - $1/' If you want to see the name of the last person, who added a message to the conversation, change the greediness of the operators like this: curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | perl -pe 's/^<title>(.*)<\/title>.*?<name>(.*?)<\/name>.*$/$2 - $1/' Show Sample Output


    46
    curl -u username:password --silent "https://mail.google.com/mail/feed/atom" | tr -d '\n' | awk -F '<entry>' '{for (i=2; i<=NF; i++) {print $i}}' | sed -n "s/<title>\(.*\)<\/title.*name>\(.*\)<\/name>.*/\2 - \1/p"
    postrational · 2009-09-07 21:56:40 6
  • Does not require input to function or complete. Number of iterations controlled by shell variable $NUM. Show Sample Output


    0
    awk 'BEGIN {a=1;b=1;for(i=0;i<'${NUM}';i++){print a;c=a+b;a=b;b=c}}'
    arcege · 2009-09-06 03:05:55 1
  • Have wc work on each file then add up the total with awk; get a 43% speed increase on RHEL over using "-exec cat|wc -l" and a 67% increase on my Ubuntu laptop (this is with 10MB of data in 767 files).


    -1
    find . -type f -name '*.c' -exec wc -l {} \; | awk '{sum+=$1} END {print sum}'
    arcege · 2009-09-04 15:51:30 3
  • needs no GNU tools, as far as I see it


    -1
    find . -exec grep foobar /dev/null {} \; | awk -F: '{print $1}' | xargs vi
    verboEse · 2009-09-03 16:02:18 0
  • saves one command. Needs GNU grep though :-(


    0
    grep -ir 'foo' * | awk -F '{print $1}' | xargs vim
    verboEse · 2009-09-03 15:58:47 0
  • The grep switches eliminate the need for awk and sed. Modifying vim with -p will show all files in separate tabs, -o in separate vim windows. Just wish it didn't hose my terminal once I exit vim!!


    3
    grep -Hrli 'foo' * | xargs vim
    dere22 · 2009-09-03 15:44:05 0
  • This will drop you into vim to edit all files that contain your grep string.


    0
    grep -ir 'foo' * | awk '{print $1}' | sed -e 's/://' | xargs vim
    elubow · 2009-09-03 15:12:27 0
  • Check which files are opened by Firefox then sort by largest size (in MB). You can see all files opened by just replacing grep to "/". Useful if you'd like to debug and check which extensions or files are taking too much memory resources in Firefox. Show Sample Output


    6
    FFPID=$(pidof firefox-bin) && lsof -p $FFPID | awk '{ if($7>0) print ($7/1024/1024)" MB -- "$9; }' | grep ".mozilla" | sort -rn
    josue · 2009-08-16 08:58:22 3
  • ( IFS=:; for i in $PATH; do echo $i; done; ) echo $PATH|sed -e 's/:/\n/g' # but the tr one is even better of course echo $PATH|xargs -d: -i echo {} # but this comes up with an extra blank line; can't figure out why and don't have the time :( echo $PATH|cut -d: --output-delimiter=' ' -f1-99 # note -- you have to hit ENTER after the first QUOTE, then type the second one. Sneaky, huh? echo $PATH | perl -l -0x3a -pe 1 # same darn extra new line; again no time to investigate echo $PATH|perl -pe 's/:/\n/g' # too obvious; clearly I'm running out of ideas :-)


    -17
    not necessarily better, but many...!
    sitaramcUnused · 2009-08-12 11:03:26 0

  • -2
    echo $PATH|awk -F: ' { for (i=1; i <= NF; i++) print $i }'
    nitins · 2009-08-12 07:56:56 1
  • ‹ First  < 9 10 11 12 13 > 

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: