Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Universal configuration monitoring and system of record for IT.
Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

May 19, 2015 - A Look At The New Commandlinefu
I've put together a short writeup on what kind of newness you can expect from the next iteration of clfu. Check it out here.
March 2, 2015 - New Management
I'm Jon, I'll be maintaining and improving clfu. Thanks to David for building such a great resource!
Hide

Top Tags

Hide

Functions

Hide

Credits

Block known dirty hosts from reaching your machine

Terminal - Block known dirty hosts from reaching your machine
wget -qO - http://infiltrated.net/blacklisted|awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}'
2009-02-18 16:08:23
User: sil
Functions: wget
33
Block known dirty hosts from reaching your machine

Blacklisted is a compiled list of all known dirty hosts (botnets, spammers, bruteforcers, etc.) which is updated on an hourly basis. This command will get the list and create the rules for you, if you want them automatically blocked, append |sh to the end of the command line. It's a more practical solution to block all and allow in specifics however, there are many who don't or can't do this which is where this script will come in handy. For those using ipfw, a quick fix would be {print "add deny ip from "$1" to any}. Posted in the sample output are the top two entries. Be advised the blacklisted file itself filters out RFC1918 addresses (10.x.x.x, 172.16-31.x.x, 192.168.x.x) however, it is advisable you check/parse the list before you implement the rules

Alternatives

There is 1 alternative - vote for the best!

Terminal - Alternatives

Know a better way?

If you can do better, submit your command here.

What others think

Great size list.

Comment by BADmd 294 weeks and 3 days ago

Can someone help me understand the "!/#|[a-z]/&&/./" portion of the awk string? Would incorporating 'sort' into the command, before feeding to iptables, improve lookup efficiency (the blacklist is not numerically ordered) or is awk accomplishing this? Thank you.

Comment by surfkid 226 weeks and 1 day ago

! means not so !/#|[a-z]/ is: ignore anything with a comment and [a-z] is ignore letters. && is an /./

so:

awk '!/# ignore anything with pound sign

|[a-z] ignore any letters

&& and

/./ show me anything with a period

awk '!/ignore_this_string|ignore_that_string/&&/show_this_one/{print $FIELD}'

its the equivalent of grep -v "#|[a-z]"

Comment by sil 226 weeks and 1 day ago

Sil, great explanation (and fast response) thank you! Thoughts on sorting the list?

Comment by surfkid 226 weeks ago

To explain things better, I figured I'd show an example: So I created a file called test with 10 lines, the first three are duplicates:

more test

99.69.161.123

99.69.161.123

99.69.161.123

99.68.231.37

99.58.203.81

99.4.136.182

99.38.203.228

99.36.16.156

99.31.123.191

99.27.202.67

Let's number them:

sed '/./=' test | sed '/./N; s/\n/ /'

1 99.69.161.123

2 99.69.161.123

3 99.69.161.123

4 99.68.231.37

5 99.58.203.81

6 99.4.136.182

7 99.38.203.228

8 99.36.16.156

9 99.31.123.191

10 99.27.202.67

So we see lines 1, 2, 3 are the same... There are a few ways to sort them uniquely. Since I began using awk, here is how to do so with awk:

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP" | "sort -u"}' test

Notice how I use the sort command inside awk? There is no reason to re-pipe it through to sort:

awk by itself

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}' test

iptables -A INPUT -s 99.69.161.123 -j DROP

iptables -A INPUT -s 99.69.161.123 -j DROP

iptables -A INPUT -s 99.69.161.123 -j DROP

iptables -A INPUT -s 99.68.231.37 -j DROP

iptables -A INPUT -s 99.58.203.81 -j DROP

iptables -A INPUT -s 99.4.136.182 -j DROP

iptables -A INPUT -s 99.38.203.228 -j DROP

iptables -A INPUT -s 99.36.16.156 -j DROP

iptables -A INPUT -s 99.31.123.191 -j DROP

iptables -A INPUT -s 99.27.202.67 -j DROP

awk with sort -u (-u is for unique instead of: more filename | sort | uniq)

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}' test | sort -u

iptables -A INPUT -s 99.27.202.67 -j DROP

iptables -A INPUT -s 99.31.123.191 -j DROP

iptables -A INPUT -s 99.36.16.156 -j DROP

iptables -A INPUT -s 99.38.203.228 -j DROP

iptables -A INPUT -s 99.4.136.182 -j DROP

iptables -A INPUT -s 99.58.203.81 -j DROP

iptables -A INPUT -s 99.68.231.37 -j DROP

iptables -A INPUT -s 99.69.161.123 -j DROP

Why bother going through this when I can just use sort inside of awk. The uglier and bloated way would be something like:

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}' filename | sort | uniq

Or even uglier:

curl www.infiltrated.net/blacklisted | ruby -ne 'puts $_ unless $_ == @prev; @prev = $_' | awk '{print "iptables -A INPUT -s "$1" -j DROP"}'

Comment by sil 226 weeks ago

This is exactly what I needed to see! Sil, you rock. The meticulous instruction was so helpful, particularly the placement of sort and its unique switch within awk versus (unnecessary) discreet use. I will use:

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP" | "sort -u"}'
Comment by surfkid 226 weeks ago

Excellent Contribution. Thank you.

Comment by LAN4N6 200 weeks and 5 days ago

Your point of view

You must be signed in to comment.