tail -f production.log | perl -ne 'if (/^Completed.in.(\d+)/){$d = int($1/1000);print "\n";$f{$d}++;for $t (sort(keys(%f))){print $t."s: ".$f{$t}."\n"}}'

filter rails log to count requests sliced in seconds

Commandline perl filter for, using a production.log from a rails app, display on realtime the count of requests grouped by "seconds to complete" (gross round, but fair enough for an oneliner) :)
Sample Output
0s: 30
1s: 43
2s: 12
5s: 1

0s: 33
1s: 43
2s: 13
5s: 1

0
By: theist
2012-02-23 14:37:33

These Might Interest You

  • Show's per IP of how many requests they did to the Apache webserver


    1
    cat /var/log/apache2/access_logs | cut -d ' ' -f 1 | uniq -c | sort -n
    basvdburg · 2013-09-02 13:04:47 1

  • -5
    echo 00:29:36 | nawk -F: '{seconds=($1*60)*60; seconds=seconds+($2*60); seconds=seconds+$3; print seconds}'
    unixmonkey3280 · 2011-11-01 10:57:45 0
  • Ever need to know why Apache is bogging down *right now*? Hate scanning Apache's Extended server-status for the longest running requests? Me, too. That's why I use this one liner to quickly find suspect web scripts that might need review. Assuming the Extended server-status is reachable at the target URL desired, this one-liner parses the output through elinks (rendering the HTML) and shows a list of active requests sorted by longest running request at the bottom of the list. I include the following fields (as noted in the header line): Seconds: How long the request is alive PID: Process ID of the request handler State: State of the request, limited to what I think are the relevant ones (GCRK_.) IP: Remote Host IP making the request Domain: Virtual Host target (HTTP/1.1 Host: header). Important for Virtual Hosting servers TYPE: HTTP verb URL: requested URL being served. Putting this in a script that runs when triggered by high load average can be quite revealing. Can also capture "forgotten" scripts being exploited such as "formmail.pl", etc. Show Sample Output


    6
    links --dump 1 http://localhost/server-status|grep ^[0-9]|awk 'BEGIN {print "Seconds, PID, State, IP, Domain, TYPE, URL\n--"} $4 !~ /[GCRK_.]/ {print $6, $2, $4, $11, $12, $13 " " $14|"sort -n"}'
    rjamestaylor · 2009-02-19 13:06:44 2
  • This is a very simple way to input a large number of seconds and get a more useful value in minutes and seconds. Avoids useless use of echo. Show Sample Output


    25
    bc <<< 'obase=60;299'
    mathias · 2011-10-21 13:15:17 1
  • This is a very simple way to input a large number of seconds and get a more useful value in minutes and seconds. Show Sample Output


    17
    echo 'obase=60;299' | bc
    unixmonkey3280 · 2011-10-21 07:55:30 2
  • *** CAREFULLY READ THE NOTES **** *** THIS DOES NOT WORK "OUT OF THE BOX" *** You'll need a few minutes of CAREFUL reading before making your own Twitter feed: In 2010 simple command line Twitter feed requests all stopped working because Twitter upgraded to SSL security. Https requests for a filtered Twitter stream feed now require a special header called "oauth_header". The benefit is that your stream feed and login info is securely encrypted. The bad news is that an "oauth_header" takes some work to build. Fortunately, four functions, imaginatively named step1, step2, step3 and step4 can be used to build a customized oauth_header for you in a few minutes. Now, go look at "step1" to start creating your own oauth_header! Show Sample Output


    10
    step1 ; step2 ; step3 ; step4 ; curl -o- --get 'https://stream.twitter.com/1/statuses/filter.json' --header "$oauth_header" --data "follow=$id"
    nixnax · 2012-03-18 21:15:04 0

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: