Commands tagged awk (348)

  • Upload images to omploader.org from the command line.

    This function uploads images to http://omploader.org and then prints out the links to the file. Some coloring can also be added to the command with: ompload() { curl -F file1=@"$1" http://omploader.org/upload|awk '/Info:|File:|Thumbnail:|BBCode:/{gsub(/<[^<]*?\/?>/,"");$1=$1;sub(/^/,"\033[0;34m");sub(/:/,"\033[0m:");print}';} Show Sample Output


    8
    ompload() { curl -# -F file1=@"$1" http://ompldr.org/upload|awk '/Info:|File:|Thumbnail:|BBCode:/{gsub(/<[^<]*?\/?>/,"");$1=$1;print}';}
    eightmillion · 2009-11-07 20:56:52 5
  • Remove all unused kernels with apt-get

    This should do the same thing and is about 70 chars shorter. Show Sample Output


    8
    aptitude remove $(dpkg -l|egrep '^ii linux-(im|he)'|awk '{print $2}'|grep -v `uname -r`)
    dbbolton · 2010-06-10 21:23:00 10
  • Find default gateway


    8
    ip route | awk '/default/{print $3}'
    rthemocap · 2010-08-10 21:16:40 51
  • Find the cover image for an album

    This bash function uses albumart.org to find the cover for an album. It returns an amazon.com url to the image. Usage: albumart [artist] [album] These arguments can be reversed and if the album name is distinct enough, it may be possible to omit the artist. The command can be extended with wget to automatically download the matching image like this: albumart(){ local x y="$@";x=$(awk '/View larger image/{gsub(/^.*largeImagePopup\(.|., .*$/,"");print;exit}' <(curl -s 'http://www.albumart.org/index.php?srchkey='${y// /+}'&itempage=1&newsearch=1&searchindex=Music'));[ -z "$x" ]&&echo "Not found."||wget "$x" -O "${y}.${x##*.}";} Show Sample Output


    7
    albumart(){ local y="$@";awk '/View larger image/{gsub(/^.*largeImagePopup\(.|., .*$/,"");print;exit}' <(curl -s 'http://www.albumart.org/index.php?srchkey='${y// /+}'&itempage=1&newsearch=1&searchindex=Music');}
    eightmillion · 2009-11-15 19:54:16 11
  • Print trending topics on Twitter


    7
    curl -s search.twitter.com | awk -F'</?[^>]+>' '/\/intra\/trend\//{print $2}'
    putnamhill · 2009-12-22 01:01:02 12
  • Remove all unused kernels with apt-get

    Note the double space: "...^ii␣␣linux-image-2..." Like 5813, but fixes two bugs: [1]This leaves the meta-packages 'linux-headers-generic' and 'linux-image-generic' alone so that automatic upgrades work correctly in the future. [2]Kernels newer than the currently running one are left alone (this can happen if you didn't reboot after installing a new kernel). I'm bummed that this took 228 characters. I'd like to see a simpler version. Show Sample Output


    7
    aptitude remove $(dpkg -l|awk '/^ii linux-image-2/{print $2}'|sed 's/linux-image-//'|awk -v v=`uname -r` 'v>$0'|sed 's/-generic//'|awk '{printf("linux-headers-%s\nlinux-headers-%s-generic\nlinux-image-%s-generic\n",$0,$0,$0)}')
    __ · 2010-12-11 11:38:15 7
  • easily strace all your apache processes

    This one-liner will use strace to attach to all of the currently running apache processes output and piped from the initial "ps auxw" command into some awk. Show Sample Output


    7
    ps auxw | grep sbin/apache | awk '{print"-p " $2}' | xargs strace
    px · 2011-03-14 21:45:22 34
  • Print one . instead of each line

    If you're running a command with a lot of output, this serves as a simple progress indicator. This avoids the need to use `/dev/null` for silencing. It works for any command that outputs lines, updates live (`fflush` avoids buffering), and is simple to understand. Show Sample Output


    7
    alias ...="awk '{fflush(); printf \".\"}' && echo \"\""
    lgarron · 2014-02-22 22:20:22 15
  • Check how far along (in %) your program is in a file

    Imagine you've started a long-running process that involves piping data, but you forgot to add the progress-bar option to a command. e.g. xz -dc bigdata.xz | complicated-processing-program > summary . This command uses lsof to see how much data xz has read from the file. lsof -o0 -o -Fo FILENAME Display offsets (-o), in decimal (-o0), in parseable form (-Fo) This will output something like: . p12607 f3 o0t45187072 . Process id (p), File Descriptor (f), Offset (o) . We stat the file to get its size stat -c %s FILENAME . Then we plug the values into awk. Split the line at the letter t: -Ft Define a variable for the file's size: -s=$(stat...) Only work on the offset line: /^o/ . Note this command was tested using the Linux version of lsof. Because it uses lsof's batch option (-F) it may be portable. . Thanks to @unhammer for the brilliant idea. Show Sample Output


    7
    F=bigdata.xz; lsof -o0 -o -Fo $F | awk -Ft -v s=$(stat -c %s $F) '/^o/{printf("%d%%\n", 100*$2/s)}'
    flatcap · 2015-09-19 22:22:43 18
  • Generates a TV noise alike output in the terminal

    Generates a TV noise alike output in the terminal. Can be combined with https://www.commandlinefu.com/commands/view/9728/make-some-powerful-pink-noise


    7
    while true;do printf "$(awk -v c="$(tput cols)" -v s="$RANDOM" 'BEGIN{srand(s);while(--c>=0){printf("\xe2\x96\\%s",sprintf("%o",150+int(10*rand())));}}')";done
    ichbins · 2020-05-08 09:55:36 209
  • Display the standard deviation of a column of numbers with awk


    6
    awk '{sum+=$1; sumsq+=$1*$1} END {print sqrt(sumsq/NR - (sum/NR)**2)}' file.dat
    kaan · 2009-03-24 21:56:40 9
  • Check which files are opened by Firefox then sort by largest size.

    Check which files are opened by Firefox then sort by largest size (in MB). You can see all files opened by just replacing grep to "/". Useful if you'd like to debug and check which extensions or files are taking too much memory resources in Firefox. Show Sample Output


    6
    FFPID=$(pidof firefox-bin) && lsof -p $FFPID | awk '{ if($7>0) print ($7/1024/1024)" MB -- "$9; }' | grep ".mozilla" | sort -rn
    josue · 2009-08-16 08:58:22 7
  • Visualizing system performance data


    6
    (echo "set terminal png;plot '-' u 1:2 t 'cpu' w linespoints;"; sudo vmstat 2 10 | awk 'NR > 2 {print NR, $13}') | gnuplot > plot.png
    grokskookum · 2009-09-23 16:40:13 12
  • Capture video of a linux desktop


    6
    ffmpeg -f x11grab -s `xdpyinfo | grep 'dimensions:'|awk '{print $2}'` -r 25 -i :0.0 -sameq /tmp/out.mpg > /root/howto/capture_screen_video_ffmpeg
    brubaker · 2010-03-27 21:31:34 5
  • drop first column of output by piping to this


    6
    tr -s ' ' | cut -d' ' -f2-
    sesom42 · 2015-05-08 12:47:09 10
  • Replace Solaris vmstat numbers with human readable format

    % cat ph-vmstat.awk # Return human readable numbers function hrnum(a) { b = a ; if (a > 1000000) { b = sprintf("%2.2fM", a/1000000) ; } else if (a > 1000) { b = sprintf("%2.2fK", a/1000) ; } return(b) ; } # Return human readable storage function hrstorage(a) { b = a ; if (a > 1024000) { b = sprintf("%2.2fG", a/1024/1024) ; } else if (a > 1024) { b = sprintf("%2.2fM", a/1024) ; } return(b) ; } OFS=" " ; $1 !~ /[0-9].*/ {print} $1 ~ /[0-9].*/ { $4 = hrstorage($4) ; $5 = hrstorage($5) ; $9 = hrnum($9) ; $10 = hrnum($10) ; $17 = hrnum($17) ; $18 = hrnum($18) ; $19 = hrnum($19) ; print ; } Show Sample Output


    5
    vmstat 1 10 | /usr/xpg4/bin/awk -f ph-vmstat.awk
    MarcoN · 2009-05-04 04:55:00 4
  • Get line number of all matches in a file


    5
    awk '/match/{print NR}' file
    inof · 2009-06-03 17:34:13 7
  • Capture screen and mic input using FFmpeg and ALSA

    Yet another x11grab using ffmpeg. I also added mic input to the capturing video stream using alsa. Yet I need to find out how to capture audio which is currently playing. Show Sample Output


    5
    ffmpeg -f alsa -itsoffset 00:00:02.000 -ac 2 -i hw:0,0 -f x11grab -s $(xwininfo -root | grep 'geometry' | awk '{print $2;}') -r 10 -i :0.0 -sameq -f mp4 -s wvga -y intro.mp4
    mohan43u · 2010-03-31 09:33:05 6
  • Show interface/ip using awk

    Interfaces like lo can be omitted from the beginning, there are probably better ways of doing this, i'm a noob at awk. Show Sample Output


    5
    ifconfig -a| awk '/^wlan|^eth|^lo/ {;a=$1;FS=":"; nextline=NR+1; next}{ if (NR==nextline) { split($2,b," ")}{ if ($2 ~ /[0-9]\./) {print a,b[1]}; FS=" "}}'
    alf · 2010-04-15 04:34:28 16
  • Get IPv4 of eth0 for use with scripts

    I've been using it in a script to build from scratch proxy servers. Show Sample Output


    5
    ifconfig eth0 | grep "inet " | cut -d ':' -f2 | awk '{print $1}'
    jaimerosario · 2010-06-29 00:06:08 4
  • Revert all modified files in an SVN repo


    5
    svn st | grep -e '^M' | awk '{print $2}' | xargs svn revert
    ethanmiller · 2010-08-11 14:24:05 3
  • Fibonacci numbers with awk

    recursive version, "pure" AWK Show Sample Output


    5
    awk 'func f(n){return(n<2?n:f(n-1)+f(n-2))}BEGIN{while(a<24){print f(a++)}}'
    unefunge · 2010-11-24 10:40:08 7
  • Lists all usernames in alphabetical order

    Save some CPU, and some PIDs. :)


    5
    awk -F ':' '{print $1 | "sort";}' /etc/passwd
    atoponce · 2011-12-20 12:46:52 3
  • Find duplicate UID in /etc/passwd

    You can use only awk


    5
    awk -F":" '!list[$3]++{print $3}' /etc/passwd
    richard · 2012-01-24 12:47:52 8
  • Found how how much memory in kB $PID is occupying in Linux

    The "proportional set size" is probably the closest representation of how much active memory a process is using in the Linux virtual memory stack. This number should also closely represent the %mem found in ps(1), htop(1), and other utilities. Show Sample Output


    5
    echo 0$(awk '/Pss/ {printf "+"$2}' /proc/$PID/smaps)|bc
    atoponce · 2013-09-26 18:20:22 10
  •  < 1 2 3 4 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Calculate days on which Friday the 13th occurs
I removed the dependency of the English language

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Jump up to any directory above the current one
Usage: upto directory

Split lossless audio (ape, flac, wav, wv) by cue file

run complex remote shell cmds over ssh, without escaping quotes
It executes commands as arguments to ssh, avoiding problematic shell expansions, without the need of writing the commands in question to a temporary file, just reading them from STDIN.

Create a mirror of a local folder, on a remote server
Create a exact mirror of the local folder "/root/files", on remote server 'remote_server' using SSH command (listening on port 22) (all files & folders on destination server/folder will be deleted)

Copy your ssh public key to a server from a machine that doesn't have ssh-copy-id
If you use Mac OS X or some other *nix variant that doesn't come with ssh-copy-id, this one-liner will allow you to add your public key to a remote machine so you can subsequently ssh to that machine without a password.

Get NFL/MLB Scores/Time
change the nfl in the url to mlb or nba to get those score/times as well

PRINT LINE the width of screen or specified using any char including Colors, Escapes and metachars
One of the first functions programmers learn is how to print a line. This is my 100% bash builtin function to do it, which makes it as optimal as a function can be. The COLUMNS environment variable is also set by bash (including bash resetting its value when you resize your term) so its very efficient. I like pretty-output in my shells and have experimented with several ways to output a line the width of the screen using a minimal amount of code. This is like version 9,000 lol. This function is what I use, though when using colors or other terminal features I create separate functions that call this one, since this is the lowest level type of function. It might be better named printl(), but since I use it so much it's more optimal to have the name contain less chars (both for my programming and for the internal workings). If you do use terminal escapes this will reset to default. $ tput sgr0 For implementation ideas, check my http://www.askapache.com/linux-unix/bash_profile-functions-advanced-shell.html

Network Folder Copy with Monitoring ( tar + nc + pv )
Transfer tar stream thru nc with pv montoiring taken from: http://www.catonmat.net/blog/unix-utilities-pipe-viewer/

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: