Commands tagged awk (348)

  • Upload images to omploader.org from the command line.

    This function uploads images to http://omploader.org and then prints out the links to the file. Some coloring can also be added to the command with: ompload() { curl -F file1=@"$1" http://omploader.org/upload|awk '/Info:|File:|Thumbnail:|BBCode:/{gsub(/<[^<]*?\/?>/,"");$1=$1;sub(/^/,"\033[0;34m");sub(/:/,"\033[0m:");print}';} Show Sample Output


    8
    ompload() { curl -# -F file1=@"$1" http://ompldr.org/upload|awk '/Info:|File:|Thumbnail:|BBCode:/{gsub(/<[^<]*?\/?>/,"");$1=$1;print}';}
    eightmillion · 2009-11-07 20:56:52 5
  • Remove all unused kernels with apt-get

    This should do the same thing and is about 70 chars shorter. Show Sample Output


    8
    aptitude remove $(dpkg -l|egrep '^ii linux-(im|he)'|awk '{print $2}'|grep -v `uname -r`)
    dbbolton · 2010-06-10 21:23:00 10
  • Find default gateway


    8
    ip route | awk '/default/{print $3}'
    rthemocap · 2010-08-10 21:16:40 51
  • Find the cover image for an album

    This bash function uses albumart.org to find the cover for an album. It returns an amazon.com url to the image. Usage: albumart [artist] [album] These arguments can be reversed and if the album name is distinct enough, it may be possible to omit the artist. The command can be extended with wget to automatically download the matching image like this: albumart(){ local x y="$@";x=$(awk '/View larger image/{gsub(/^.*largeImagePopup\(.|., .*$/,"");print;exit}' <(curl -s 'http://www.albumart.org/index.php?srchkey='${y// /+}'&itempage=1&newsearch=1&searchindex=Music'));[ -z "$x" ]&&echo "Not found."||wget "$x" -O "${y}.${x##*.}";} Show Sample Output


    7
    albumart(){ local y="$@";awk '/View larger image/{gsub(/^.*largeImagePopup\(.|., .*$/,"");print;exit}' <(curl -s 'http://www.albumart.org/index.php?srchkey='${y// /+}'&itempage=1&newsearch=1&searchindex=Music');}
    eightmillion · 2009-11-15 19:54:16 11
  • Print trending topics on Twitter


    7
    curl -s search.twitter.com | awk -F'</?[^>]+>' '/\/intra\/trend\//{print $2}'
    putnamhill · 2009-12-22 01:01:02 12
  • Remove all unused kernels with apt-get

    Note the double space: "...^ii␣␣linux-image-2..." Like 5813, but fixes two bugs: [1]This leaves the meta-packages 'linux-headers-generic' and 'linux-image-generic' alone so that automatic upgrades work correctly in the future. [2]Kernels newer than the currently running one are left alone (this can happen if you didn't reboot after installing a new kernel). I'm bummed that this took 228 characters. I'd like to see a simpler version. Show Sample Output


    7
    aptitude remove $(dpkg -l|awk '/^ii linux-image-2/{print $2}'|sed 's/linux-image-//'|awk -v v=`uname -r` 'v>$0'|sed 's/-generic//'|awk '{printf("linux-headers-%s\nlinux-headers-%s-generic\nlinux-image-%s-generic\n",$0,$0,$0)}')
    __ · 2010-12-11 11:38:15 7
  • easily strace all your apache processes

    This one-liner will use strace to attach to all of the currently running apache processes output and piped from the initial "ps auxw" command into some awk. Show Sample Output


    7
    ps auxw | grep sbin/apache | awk '{print"-p " $2}' | xargs strace
    px · 2011-03-14 21:45:22 34
  • Print one . instead of each line

    If you're running a command with a lot of output, this serves as a simple progress indicator. This avoids the need to use `/dev/null` for silencing. It works for any command that outputs lines, updates live (`fflush` avoids buffering), and is simple to understand. Show Sample Output


    7
    alias ...="awk '{fflush(); printf \".\"}' && echo \"\""
    lgarron · 2014-02-22 22:20:22 15
  • Check how far along (in %) your program is in a file

    Imagine you've started a long-running process that involves piping data, but you forgot to add the progress-bar option to a command. e.g. xz -dc bigdata.xz | complicated-processing-program > summary . This command uses lsof to see how much data xz has read from the file. lsof -o0 -o -Fo FILENAME Display offsets (-o), in decimal (-o0), in parseable form (-Fo) This will output something like: . p12607 f3 o0t45187072 . Process id (p), File Descriptor (f), Offset (o) . We stat the file to get its size stat -c %s FILENAME . Then we plug the values into awk. Split the line at the letter t: -Ft Define a variable for the file's size: -s=$(stat...) Only work on the offset line: /^o/ . Note this command was tested using the Linux version of lsof. Because it uses lsof's batch option (-F) it may be portable. . Thanks to @unhammer for the brilliant idea. Show Sample Output


    7
    F=bigdata.xz; lsof -o0 -o -Fo $F | awk -Ft -v s=$(stat -c %s $F) '/^o/{printf("%d%%\n", 100*$2/s)}'
    flatcap · 2015-09-19 22:22:43 18
  • Generates a TV noise alike output in the terminal

    Generates a TV noise alike output in the terminal. Can be combined with https://www.commandlinefu.com/commands/view/9728/make-some-powerful-pink-noise


    7
    while true;do printf "$(awk -v c="$(tput cols)" -v s="$RANDOM" 'BEGIN{srand(s);while(--c>=0){printf("\xe2\x96\\%s",sprintf("%o",150+int(10*rand())));}}')";done
    ichbins · 2020-05-08 09:55:36 209
  • Display the standard deviation of a column of numbers with awk


    6
    awk '{sum+=$1; sumsq+=$1*$1} END {print sqrt(sumsq/NR - (sum/NR)**2)}' file.dat
    kaan · 2009-03-24 21:56:40 9
  • Check which files are opened by Firefox then sort by largest size.

    Check which files are opened by Firefox then sort by largest size (in MB). You can see all files opened by just replacing grep to "/". Useful if you'd like to debug and check which extensions or files are taking too much memory resources in Firefox. Show Sample Output


    6
    FFPID=$(pidof firefox-bin) && lsof -p $FFPID | awk '{ if($7>0) print ($7/1024/1024)" MB -- "$9; }' | grep ".mozilla" | sort -rn
    josue · 2009-08-16 08:58:22 7
  • Visualizing system performance data


    6
    (echo "set terminal png;plot '-' u 1:2 t 'cpu' w linespoints;"; sudo vmstat 2 10 | awk 'NR > 2 {print NR, $13}') | gnuplot > plot.png
    grokskookum · 2009-09-23 16:40:13 12
  • Capture video of a linux desktop


    6
    ffmpeg -f x11grab -s `xdpyinfo | grep 'dimensions:'|awk '{print $2}'` -r 25 -i :0.0 -sameq /tmp/out.mpg > /root/howto/capture_screen_video_ffmpeg
    brubaker · 2010-03-27 21:31:34 5
  • drop first column of output by piping to this


    6
    tr -s ' ' | cut -d' ' -f2-
    sesom42 · 2015-05-08 12:47:09 10
  • Replace Solaris vmstat numbers with human readable format

    % cat ph-vmstat.awk # Return human readable numbers function hrnum(a) { b = a ; if (a > 1000000) { b = sprintf("%2.2fM", a/1000000) ; } else if (a > 1000) { b = sprintf("%2.2fK", a/1000) ; } return(b) ; } # Return human readable storage function hrstorage(a) { b = a ; if (a > 1024000) { b = sprintf("%2.2fG", a/1024/1024) ; } else if (a > 1024) { b = sprintf("%2.2fM", a/1024) ; } return(b) ; } OFS=" " ; $1 !~ /[0-9].*/ {print} $1 ~ /[0-9].*/ { $4 = hrstorage($4) ; $5 = hrstorage($5) ; $9 = hrnum($9) ; $10 = hrnum($10) ; $17 = hrnum($17) ; $18 = hrnum($18) ; $19 = hrnum($19) ; print ; } Show Sample Output


    5
    vmstat 1 10 | /usr/xpg4/bin/awk -f ph-vmstat.awk
    MarcoN · 2009-05-04 04:55:00 4
  • Get line number of all matches in a file


    5
    awk '/match/{print NR}' file
    inof · 2009-06-03 17:34:13 7
  • Capture screen and mic input using FFmpeg and ALSA

    Yet another x11grab using ffmpeg. I also added mic input to the capturing video stream using alsa. Yet I need to find out how to capture audio which is currently playing. Show Sample Output


    5
    ffmpeg -f alsa -itsoffset 00:00:02.000 -ac 2 -i hw:0,0 -f x11grab -s $(xwininfo -root | grep 'geometry' | awk '{print $2;}') -r 10 -i :0.0 -sameq -f mp4 -s wvga -y intro.mp4
    mohan43u · 2010-03-31 09:33:05 6
  • Show interface/ip using awk

    Interfaces like lo can be omitted from the beginning, there are probably better ways of doing this, i'm a noob at awk. Show Sample Output


    5
    ifconfig -a| awk '/^wlan|^eth|^lo/ {;a=$1;FS=":"; nextline=NR+1; next}{ if (NR==nextline) { split($2,b," ")}{ if ($2 ~ /[0-9]\./) {print a,b[1]}; FS=" "}}'
    alf · 2010-04-15 04:34:28 16
  • Get IPv4 of eth0 for use with scripts

    I've been using it in a script to build from scratch proxy servers. Show Sample Output


    5
    ifconfig eth0 | grep "inet " | cut -d ':' -f2 | awk '{print $1}'
    jaimerosario · 2010-06-29 00:06:08 4
  • Revert all modified files in an SVN repo


    5
    svn st | grep -e '^M' | awk '{print $2}' | xargs svn revert
    ethanmiller · 2010-08-11 14:24:05 3
  • Fibonacci numbers with awk

    recursive version, "pure" AWK Show Sample Output


    5
    awk 'func f(n){return(n<2?n:f(n-1)+f(n-2))}BEGIN{while(a<24){print f(a++)}}'
    unefunge · 2010-11-24 10:40:08 7
  • Lists all usernames in alphabetical order

    Save some CPU, and some PIDs. :)


    5
    awk -F ':' '{print $1 | "sort";}' /etc/passwd
    atoponce · 2011-12-20 12:46:52 3
  • Find duplicate UID in /etc/passwd

    You can use only awk


    5
    awk -F":" '!list[$3]++{print $3}' /etc/passwd
    richard · 2012-01-24 12:47:52 8
  • Found how how much memory in kB $PID is occupying in Linux

    The "proportional set size" is probably the closest representation of how much active memory a process is using in the Linux virtual memory stack. This number should also closely represent the %mem found in ps(1), htop(1), and other utilities. Show Sample Output


    5
    echo 0$(awk '/Pss/ {printf "+"$2}' /proc/$PID/smaps)|bc
    atoponce · 2013-09-26 18:20:22 10
  •  < 1 2 3 4 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands

Check These Out

Pack up some files into a tarball on a remote server without writing to the local filesystem
I recently found myself with a filesystem I couldn't write to and a bunch of files I had to get the hell out of dodge, preferably not one at a time. This command makes it possible to pack a bunch of files into a single archive and write it to a remote server.

pretend to be busy in office to enjoy a cup of coffee
just make some data scrolling off the terminal. wow.

Print a row of characters across the terminal
Pure Bash This will print a row of characters the width of the screen without using any external executables. In some cases, COLUMNS may not be set. Here is an alternative that uses tput to generate a default if that's the case. And it still avoids using tr. $ printf -v row "%${COLUMNS:-$(tput cols)}s"; echo ${row// /#} The only disadvantage to either one is that they create a variable.

Find the package that installed a command

List the Sizes of Folders and Directories
I wanted an easy way to list out the sizes of directories and all of the contents of those directories recursively.

Capitalize first letter of each word in a string - A ruby alternative
"-n" loops around ; "-e" executes the given quoted string ; "$_" is the current line ; "split" creates an array on white space; each item of the array is "collected" to be then "capitalized" ; the array is "joined" back into a string.

Which processes are listening on a specific port (e.g. port 80)
swap out "80" for your port of interest. Can use port number or named ports e.g. "http"

Displays the number of unread messages on your gmail at the top right corner of your terminal
Checks your gmail account every 30 seconds and display the number of new messages in the top right corner of the terminal. A kind of CLI "Gmail notifier" if you will. :-) This is a mashup of http://www.commandlinefu.com/commands/view/7916/put-a-console-clock-in-top-right-corner and http://www.commandlinefu.com/commands/view/3386/check-your-unread-gmail-from-the-command-line

Show Directories in the PATH Which does NOT Exist
I often need to know of my directory in the PATH, which one DOES NOT exist. This command answers that question * This command uses only bash's built-in commands * The parentheses spawn a new sub shell to prevent the modification of the IFS (input field separator) variable in the current shell

Stop long commands wrapping around and over-writing itself in the Bash shell
add the command either in /etc/profile or ~/.bash_profile so that this is available to your shell.

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: