awk 'BEGIN{size=5} {mod=NR%size; if(NR<=size){count++}else{sum-=array[mod]};sum+=$1;array[mod]=$1;print sum/count}' file.dat

Awk: Perform a rolling average on a column of data

Sometimes jittery data hides trends, performing a rolling average can give a clearer view.

3
By: mungewell
2009-05-29 00:07:24

These Might Interest You

  • This is an on-line algorithm for calculating the mean value for numbers in a column. Also known as "running average" or "moving average".


    6
    awk '{avg += ($1 - avg) / NR;} END { print avg; }'
    ashawley · 2009-09-10 17:06:03 1
  • Computes a columns average in a file. Input parameters = column number and optional pattern. Show Sample Output


    0
    function avg { awk "/$2/{sum += \$$1; lc += 1;} END {printf \"Average over %d lines: %f\n\", lc, sum/lc}"; }
    vimzard · 2010-02-18 10:20:22 0
  • grep 'HOME.*' data.txt | awk '{print $2}' | awk '{FS="/"}{print $NF}' OR awk '/HOME/ {print $2}' data.txt | awk -F'/' '{print $NF}' In this example, we are having a text file that is having several entries like: --- c1 c2 c3 c4 this is some data HOME /dir1/dir2/.../dirN/somefile1.xml HOME /dir1/dir2/somefile2.xml some more data --- for lines starting with HOME, we are extracting the second field that is a 'file path with file name', and from that we need to get the filename only and ignore the slash delimited path. The output would be: somefile1.xml somefile2.xml (In case you give a -ive - pls give the reasons as well and enlighten the souls :-) )


    -3
    grep 'HOME.*' data.txt | awk '{print $2}' | awk '{FS="/"}{print $NF}' OR USE ALTERNATE WAY awk '/HOME/ {print $2}' data.txt | awk -F'/' '{print $NF}'
    rommelsharma · 2009-03-05 07:28:26 7

  • 1
    for i in `ls /var/log/sa/|grep -E "sa[0-9][0-9]"`;do echo -ne "$i -- ";sar -r -f /var/log/sa/$i|awk '{ printf "%3.2f\n",($4-$6-$7)*100/(3+$4)}'|grep -Eiv "average|linux|^ --|0.00|^-" |awk '{sum+=$1 }END{printf "Average = %3.2f%%\n",sum/NR}';done
    omatictost · 2012-10-25 17:00:58 0
  • Here's an annotated version of the command, using full-names instead of aliases. It is exactly equivalent to the short-hand version. # Recursively list all the files in the current directory. Get-ChildItem -Recurse | # Filter out the sub-directories themselves. Where-Object { return -not $_.PsIsContainer; } | # Group the resulting files by their extensions. Group-Object Extension | # Pluck the Name and Count properties of each group and define # a custom expression that calculates the average of the sizes # of the files in that group. # The back-tick is a line-continuation character. Select-Object ` Name, Count, @{ Name = 'Average'; Expression = { # Average the Length (sizes) of the files in the current group. return ($_.Group | Measure-Object -Average Length).Average; } } | # Format the results in a tabular view, automatically adjusted to # widths of the values in the columns. Format-Table -AutoSize ` @{ # Rename the Name property to something more sensible. Name = 'Extension'; Expression = { return $_.Name; } }, Count, @{ # Format the Average property to display KB instead of bytes # and use a formatting string to show it rounded to two decimals. Name = 'Average Size (KB)'; # The "1KB" is a built-in constant which is equal to 1024. Expression = { return $_.Average / 1KB }; FormatString = '{0:N2}' } Show Sample Output


    0
    ls -r | ?{-not $_.psiscontainer} | group extension | select name, count, @{n='average'; e={($_.group | measure -a length).average}} | ft -a @{n='Extension'; e={$_.name}}, count, @{n='Average Size (KB)'; e={$_.average/1kb}; f='{0:N2}'}
    brianpeiris · 2012-03-13 17:58:10 2
  • Removes duplicates in the specified field/column while outputting entire lines. An elegant command for processing tab (or otherwise) delimited data.


    1
    awk '!array[$1]++' file.txt
    bede · 2012-08-23 21:04:51 0

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: