Commands using awk (1,384)

  • This should do the same thing and is about 70 chars shorter. Show Sample Output


    8
    aptitude remove $(dpkg -l|egrep '^ii linux-(im|he)'|awk '{print $2}'|grep -v `uname -r`)
    dbbolton · 2010-06-10 21:23:00 4

  • 8
    awk 'NR==4'
    Aparicio · 2010-07-08 21:04:04 0
  • If your wc does not have the -L option, you can use awk.


    8
    awk '(length > n) {n = length} END {print n}'
    putnamhill · 2011-08-15 13:10:38 1
  • credit to tumblr engineering blog @ http://engineering.tumblr.com/ Show Sample Output


    8
    netstat -tn | awk 'NR>2 {print $6}' | sort | uniq -c | sort -rn
    evandrix · 2011-09-12 23:29:39 0
  • This command uses awk(1) to print all lines between two known line numbers in a file. Useful for seeing output in a log file, where the line numbers are known. The above command will print all lines between, and including, lines 3 and 6.


    8
    awk 'NR >= 3 && NR <= 6' /path/to/file
    atoponce · 2011-12-14 14:28:56 3

  • 8
    history | awk '{print $2}' | sort | uniq -c | sort -rn | head
    ken_fallon · 2012-01-05 22:23:25 1
  • Certain Flash video players (e.g. Youtube) write their video streams to disk in /tmp/ , but the files are unlinked. i.e. the player creates the file and then immediately deletes the filename (unlinking files in this way makes it hard to find them, and/or ensures their cleanup if the browser or plugin should crash etc.) But as long as the flash plugin's process runs, a file descriptor remains in its /proc/ hierarchy, from which we (and the player) still have access to the file. The method above worked nicely for me when I had 50 tabs open with Youtube videos and didn't want to have to re-download them all with some tool.


    8
    lsof -n -P|grep FlashXX|awk '{ print "/proc/" $2 "/fd/" substr($4, 1, length($4)-1) }'|while read f;do newname=$(exiftool -FileModifyDate -FileType -t -d %Y%m%d%H%M%S $f|cut -f2|tr '\n' '.'|sed 's/\.$//');echo "$f -> $newname";cp $f ~/Vids/$newname;done
    mhs · 2012-02-25 01:49:45 1

  • 8
    awk '{s+=$1}END{print s}' <file>
    evandrix · 2013-01-10 03:16:48 0
  • The $2, $3, $4 fields are arbitrary but note that the first field starts from $2 and the last field is $NF-1. This is due to the fact that the leading and trailing quotes are treated as field delimiters. Show Sample Output


    7
    awk -F'^"|", "|"$' '{ print $2,$3,$4 }' file.csv
    SiegeX · 2009-02-16 21:32:46 0
  • This command is much quicker than the alternative of "sort | uniq -c | sort -n". Show Sample Output


    7
    cat $(ls -tr | tail -1) | awk '{ a[$1] += 1; } END { for(i in a) printf("%d, %s\n", a[i], i ); }' | sort -n | tail -25
    oremj · 2009-03-06 17:50:29 0
  • It will return a ranked list of your most commonly-entered commands using your command history Show Sample Output


    7
    history|awk '{print $2}'|awk 'BEGIN {FS="|"} {print $1}'|sort|uniq -c|sort -r
    kayowas · 2009-04-05 13:40:56 5

  • 7
    pdfinfo Virtualization_A_Beginner_Guide.pdf | awk /Pages/
    ncaio · 2009-05-31 15:45:38 2
  • fancy command line ncdu clone Show Sample Output


    7
    for i in `du --max-depth=1 $HOME | sort -n -r | awk '{print $1 ":" $2}'`; do size=`echo $i | awk -F: '{print $1}'`; dir=`echo $i | awk -F: '{print $NF}'`; size2=$(($size/1024)); echo "$size2 MB used by $dir"; done | head -n 10
    tuxifier · 2009-06-02 21:27:48 2
  • Uses the data in the /proc system, provided by the acpid, to find out the CPU temperature. Can be run on systems without lm-sensors installed as well. Show Sample Output


    7
    echo `date +%m/%d/%y%X |awk '{print $1;}' `" => "` cat /proc/acpi/thermal_zone/THRM/temperature | awk '{print $2, $3;}'` >> datetmp.log
    ninadsp · 2009-08-24 21:26:29 2
  • - convert unixtime to human-readable with awk - useful to read logfiles with unix-timestamps, f.e. squid-log: sudo tail -f /var/log/squid3/access.log | awk '{ print strftime("%c ", $1) $0; } Show Sample Output


    7
    echo 1234567890 | awk '{ print strftime("%c", $0); }'
    alvinx · 2009-08-25 09:37:54 0
  • Show apps that use internet connection at the moment. Can be used to discover what programms create internet traffic. Skip the part after awk to get more details, though it will not work showing only unique processes. This version will work with other languages such as Spanish and Portuguese, if the word for "ESTABLISHED" still contain the fragment "STAB"(e.g. "ESTABELECIDO") Show Sample Output


    7
    netstat -lantp | grep -i stab | awk -F/ '{print $2}' | sort | uniq
    ProMole · 2009-09-19 14:54:31 0
  • The original version gives an error, here is the correct output


    7
    ps -eo user,pcpu,pmem | tail -n +2 | awk '{num[$1]++; cpu[$1] += $2; mem[$1] += $3} END{printf("NPROC\tUSER\tCPU\tMEM\n"); for (user in cpu) printf("%d\t%s\t%.2f\t%.2f\n",num[user], user, cpu[user], mem[user]) }'
    georgz · 2009-10-29 12:49:01 0

  • 7
    curl -s search.twitter.com | awk -F'</?[^>]+>' '/\/intra\/trend\//{print $2}'
    putnamhill · 2009-12-22 01:01:02 1
  • see the TIME_WAIT and ESTABLISHED nums of the network Show Sample Output


    7
    netstat -n | awk '/^tcp/ {++B[$NF]} END {for(a in B) print a, B[a]}'
    healthly · 2010-04-24 12:11:52 0
  • Displays a connection histogram of active tcp connections. Works even better under an alias. Thanks @Areis1 for sharing this one.


    7
    netstat -an | grep ESTABLISHED | awk '\''{print $5}'\'' | awk -F: '\''{print $1}'\'' | sort | uniq -c | awk '\''{ printf("%s\t%s\t",$2,$1); for (i = 0; i < $1; i++) {printf("*")}; print ""}'\''
    mramos · 2010-07-09 00:25:45 2

  • 7
    ip route | awk '/default/{print $3}'
    rthemocap · 2010-08-10 21:16:40 1
  • Note the double space: "...^ii␣␣linux-image-2..." Like 5813, but fixes two bugs: [1]This leaves the meta-packages 'linux-headers-generic' and 'linux-image-generic' alone so that automatic upgrades work correctly in the future. [2]Kernels newer than the currently running one are left alone (this can happen if you didn't reboot after installing a new kernel). I'm bummed that this took 228 characters. I'd like to see a simpler version. Show Sample Output


    7
    aptitude remove $(dpkg -l|awk '/^ii linux-image-2/{print $2}'|sed 's/linux-image-//'|awk -v v=`uname -r` 'v>$0'|sed 's/-generic//'|awk '{printf("linux-headers-%s\nlinux-headers-%s-generic\nlinux-image-%s-generic\n",$0,$0,$0)}')
    __ · 2010-12-11 11:38:15 3
  • This one-liner will use strace to attach to all of the currently running apache processes output and piped from the initial "ps auxw" command into some awk. Show Sample Output


    7
    ps auxw | grep sbin/apache | awk '{print"-p " $2}' | xargs strace
    px · 2011-03-14 21:45:22 7
  • limite = threshold Show Sample Output


    7
    limite="5";load5=$(awk '{print $1}' /proc/loadavg);echo "http://chart.apis.google.com/chart?chxr=0,0,5&chxt=y&chs=700x240&cht=gm&chds=0,"$limite"&chd=t:"$load5"&chl="$load5"&chtt=$(hostname)+load+average"
    ncaio · 2011-04-02 17:55:24 4
  • Imagine you've started a long-running process that involves piping data, but you forgot to add the progress-bar option to a command. e.g. xz -dc bigdata.xz | complicated-processing-program > summary . This command uses lsof to see how much data xz has read from the file. lsof -o0 -o -Fo FILENAME Display offsets (-o), in decimal (-o0), in parseable form (-Fo) This will output something like: . p12607 f3 o0t45187072 . Process id (p), File Descriptor (f), Offset (o) . We stat the file to get its size stat -c %s FILENAME . Then we plug the values into awk. Split the line at the letter t: -Ft Define a variable for the file's size: -s=$(stat...) Only work on the offset line: /^o/ . Note this command was tested using the Linux version of lsof. Because it uses lsof's batch option (-F) it may be portable. . Thanks to @unhammer for the brilliant idea. Show Sample Output


    7
    F=bigdata.xz; lsof -o0 -o -Fo $F | awk -Ft -v s=$(stat -c %s $F) '/^o/{printf("%d%%\n", 100*$2/s)}'
    flatcap · 2015-09-19 22:22:43 1
  • ‹ First  < 2 3 4 5 6 >  Last ›

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands


Check These Out

Ask user to confirm
Returns true if user presses the key. Use it like $ Confirm "Continue" && do action

Watch movies in your terminal
requires mplayer

Filter IP's in apache access logs based on use
Show's per IP of how many requests they did to the Apache webserver

Add a progress counter to loop (see sample output)
For this hack you need following function: $ finit() { count=$#; current=1; for i in "$@" ; do echo $current $count; echo $i; current=$((current + 1)); done; } and alias: $ alias fnext='read cur total && echo -n "[$cur/$total] " && read' Inspired by CMake progress counters.

list files recursively by size

Count the total amount of hours of your music collection
First the find command finds all files in your current directory (.). This is piped to xargs to be able to run the next shell pipeline in parallel. The xargs -P argument specifies how many processes you want to run in parallel, you can set this higher than your core count as the duration reading is mainly IO bound. The -print0 and -0 arguments of find and xargs respectively are used to easily handle files with spaces or other special characters. A subshell is executed by xargs to have a shell pipeline for each file that is found by find. This pipeline extracts the duration and converts it to a format easily parsed by awk. ffmpeg reads the file and prints a lot of information about it, grep extracts the duration line. cut and sed cut out the time information, and tr converts the last . to a : to make it easier to split by awk. awk is a specialized programming language for use in shell scripts. Here we use it to split the time elements in 4 variables and add them up.

change exif data in all jpeg's
this takes every jpg in the current directory and changes the exif data for manufactur and model. in my case i change it to LOMO LC-A because my scanner puts his data in there :]

Download all MegaTokyo strips
A simple script for download all the MegaTokyo strips from the first to the last one

print contents of file from first match of regex to end of file
Search in "filename" for the first line to match regex, and print to stdout from the matching line to the end of the file.

Add all unversioned files to svn


Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: