Print summary of referers with X amount of occurances

awk -F\" '{print $4}' *.log | grep -v "eviljaymz\|\-" | sort | uniq -c | awk -F\ '{ if($1>500) print $1,$2;}' | sort -n
This prints a summary of your referers from your logs as long as they occurred a certain number of times (in this case 500). The grep command excludes the terms, I add this in to remove results Im not interested in.
Sample Output
jaymz@bowser:~/Desktop/tmp$ awk -F\" '{print $4}' *.log | grep -v "eviljaymz\|\-" | sort | uniq -c | awk -F\  '{ if($1>500) print $1,$2;}' | sort -n
517 http://kyberia.sk/id/63756
621 http://kyberia.sk/id/15
628 http://bdelive.fr/shoutbox_view.php?0?auto_refresh=1
644 http://digg.com/all/upcoming/most
658 http://winkftp.free.fr/minichat/minichatone.php
690 http://loading.se/forum.php?thread_id=125&page=412
704 http://www.reddit.com/new/
715 http://www.okoun.cz/boards/komixove_stripy
726 http://www.skyscrapercity.com/showthread.php?t=521993&page=343
759 http://www.reddit.com/r/programming/comments/7zena/reasons_why_people_who_work_with_computers_seem/
760 http://www.kalerab.sk/forum.php?forumid=1243
787 http://www.graphilla.com/viewtopic.php?t=15983&start=1000
819 http://www.google.com/reader/view/
878 http://www.reddit.com/?count=25&after=t3_7z5oj
884 http://www.diskusjon.no/index.php?showtopic=690294&st=8680
998 http://www.reddit.com/top/
1151 http://www.imilovice.cz/main/forum.php?action=showthread&threadid=20192
1239 http://www.ziapps.com/
1259 http://www.jimmyr.com/
1369 http://www.nyx.cz/index.php?l=topic;id=64
1381 http://www.stumbleupon.com/toolbar/
2559 http://twitturls.com/
2763 http://www.reddit.com/
2897 http://www.waarmaarraar.nl/opmerkelijkbeeld/0/GO/0/opmerkelijke_beelden.html
2994 http://www.reddit.com/r/pics/
4445 http://www.reddit.com/r/programming/
8222 http://www.bdelive.fr/shoutbox_view.php?0?auto_refresh=1
24220 http://www.plurk.com/getWidget?uid=423872&h=375&w=190&u_info=1&bg=cf682f&tl=cae7fd

1
By: jaymzcd
2009-05-05 22:21:04

These Might Interest You

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: