summarize a list of IP addresses, verifying IP address and giving counts for each IP found

function summaryIP() { < $1 awk '{print $1}' | while read ip ; do verifyIP ${ip} && echo ${ip}; done | awk '{ip_array[$1]++} END { for (ip in ip_array) printf("%5d\t%s\n", ip_array[ip], ip)}' | sort -rn; }
Working with lists of IP addresses it is sometimes useful to summarize a count of how many times an IP address appears in the file. This example, summarizeIP, uses another function "verifyIP" previously defined in commandlinefu.com to ensure only valid IP addresses get counted. The summary list is presented in count order starting with highest count.
Sample Output
Example file containing a list of IPs:
$ nl list_of_IPs
     1  1.2.3.4
     2  5.6.7.8
     3  9.10.11.12
     4  13.14.15.16
     5  17.18.19.20
     6  21.22.23.24
     7  25.26.27.28
     8  some junk which should not be here
     9  29.30.31.32
    10  9.10.11.12
    11  9.10.11.12
    12  9.10.11.12
    13  1001.4.5.6

Having defined functions summaryIP and verifyIP (from previous commandlinefu.com posting):

$ summaryIP list_of_IPs
    4   9.10.11.12
    1   5.6.7.8
    1   29.30.31.32
    1   25.26.27.28
    1   21.22.23.24
    1   17.18.19.20
    1   13.14.15.16
    1   1.2.3.4
    

1
By: mpb
2015-05-01 16:45:05

What Others Think

OK, I can see a few things I can tidy :-) . 'read' is greedy, so everything gets put in ip. We can be smarter using a second variable: ... | while read ip X; do ip contains the first field as delimited by $IFS and X contains the rest of the line. That means we don't need the first awk any more. . We can drop the braces around ${ip} (I think you've spent too long in some other scripting language) However, it's best to quote the filename "$1". As for the second awk, I'd go "old-school" and replace it with sort, uniq, sort. . That leaves: function summaryIP() { while read ip x; do verifyIP $ip && echo $ip; done < "$1" | sort | uniq -c | sort -rn; } 111 characters, down from 206 :-) . For extra points, replace "$1" with "${1:-/dev/stdin}" to allow both: summaryIP FILENAME cat FILENAME | summaryIP . function summaryIP() { while read ip x; do verifyIP $ip && echo $ip; done < "${1:-/dev/stdin}" | sort | uniq -c | sort -rn; }
flatcap · 224 weeks and 5 days ago

What do you think?

Any thoughts on this command? Does it work on your machine? Can you do the same thing with only 14 characters?

You must be signed in to comment.

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.

Share Your Commands



Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for: