Hide

What's this?

commandlinefu.com is the place to record those command-line gems that you return to again and again.

Delete that bloated snippets file you've been using and share your personal repository with the world. That way others can gain from your CLI wisdom and you from theirs too. All commands can be commented on, discussed and voted up or down.


If you have a new feature suggestion or find a bug, please get in touch via http://commandlinefu.uservoice.com/

Get involved!

You can sign-in using OpenID credentials, or register a traditional username and password.

First-time OpenID users will be automatically assigned a username which can be changed after signing in.

Hide

Stay in the loop…

Follow the Tweets.

Every new command is wrapped in a tweet and posted to Twitter. Following the stream is a great way of staying abreast of the latest commands. For the more discerning, there are Twitter accounts for commands that get a minimum of 3 and 10 votes - that way only the great commands get tweeted.

» http://twitter.com/commandlinefu
» http://twitter.com/commandlinefu3
» http://twitter.com/commandlinefu10

Subscribe to the feeds.

Use your favourite RSS aggregator to stay in touch with the latest commands. There are feeds mirroring the 3 Twitter streams as well as for virtually every other subset (users, tags, functions,…):

Subscribe to the feed for:

Hide

News

2011-03-12 - Confoo 2011 presentation
Slides are available from the commandlinefu presentation at Confoo 2011: http://presentations.codeinthehole.com/confoo2011/
2011-01-04 - Moderation now required for new commands
To try and put and end to the spamming, new commands require moderation before they will appear on the site.
2010-12-27 - Apologies for not banning the trolls sooner
Have been away from the interwebs over Christmas. Will be more vigilant henceforth.
2010-09-24 - OAuth and pagination problems fixed
Apologies for the delay in getting Twitter's OAuth supported. Annoying pagination gremlin also fixed.
Hide

Tags

Hide

Functions

Block known dirty hosts from reaching your machine

Terminal - Block known dirty hosts from reaching your machine
wget -qO - http://infiltrated.net/blacklisted|awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}'
2009-02-18 16:08:23
User: sil
Functions: wget
33
Block known dirty hosts from reaching your machine

Blacklisted is a compiled list of all known dirty hosts (botnets, spammers, bruteforcers, etc.) which is updated on an hourly basis. This command will get the list and create the rules for you, if you want them automatically blocked, append |sh to the end of the command line. It's a more practical solution to block all and allow in specifics however, there are many who don't or can't do this which is where this script will come in handy. For those using ipfw, a quick fix would be {print "add deny ip from "$1" to any}. Posted in the sample output are the top two entries. Be advised the blacklisted file itself filters out RFC1918 addresses (10.x.x.x, 172.16-31.x.x, 192.168.x.x) however, it is advisable you check/parse the list before you implement the rules

Alternatives

There is 1 alternative - vote for the best!

Terminal - Alternatives

Know a better way?

If you can do better, submit your command here.

What others think

Great size list.

Comment by BADmd 246 weeks and 3 days ago

Can someone help me understand the "!/#|[a-z]/&&/./" portion of the awk string? Would incorporating 'sort' into the command, before feeding to iptables, improve lookup efficiency (the blacklist is not numerically ordered) or is awk accomplishing this? Thank you.

Comment by surfkid 178 weeks and 1 day ago

! means not so !/#|[a-z]/ is: ignore anything with a comment and [a-z] is ignore letters. && is an /./

so:

awk '!/# ignore anything with pound sign

|[a-z] ignore any letters

&& and

/./ show me anything with a period

awk '!/ignore_this_string|ignore_that_string/&&/show_this_one/{print $FIELD}'

its the equivalent of grep -v "#|[a-z]"

Comment by sil 178 weeks and 1 day ago

Sil, great explanation (and fast response) thank you! Thoughts on sorting the list?

Comment by surfkid 178 weeks ago

To explain things better, I figured I'd show an example: So I created a file called test with 10 lines, the first three are duplicates:

more test

99.69.161.123

99.69.161.123

99.69.161.123

99.68.231.37

99.58.203.81

99.4.136.182

99.38.203.228

99.36.16.156

99.31.123.191

99.27.202.67

Let's number them:

sed '/./=' test | sed '/./N; s/\n/ /'

1 99.69.161.123

2 99.69.161.123

3 99.69.161.123

4 99.68.231.37

5 99.58.203.81

6 99.4.136.182

7 99.38.203.228

8 99.36.16.156

9 99.31.123.191

10 99.27.202.67

So we see lines 1, 2, 3 are the same... There are a few ways to sort them uniquely. Since I began using awk, here is how to do so with awk:

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP" | "sort -u"}' test

Notice how I use the sort command inside awk? There is no reason to re-pipe it through to sort:

awk by itself

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}' test

iptables -A INPUT -s 99.69.161.123 -j DROP

iptables -A INPUT -s 99.69.161.123 -j DROP

iptables -A INPUT -s 99.69.161.123 -j DROP

iptables -A INPUT -s 99.68.231.37 -j DROP

iptables -A INPUT -s 99.58.203.81 -j DROP

iptables -A INPUT -s 99.4.136.182 -j DROP

iptables -A INPUT -s 99.38.203.228 -j DROP

iptables -A INPUT -s 99.36.16.156 -j DROP

iptables -A INPUT -s 99.31.123.191 -j DROP

iptables -A INPUT -s 99.27.202.67 -j DROP

awk with sort -u (-u is for unique instead of: more filename | sort | uniq)

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}' test | sort -u

iptables -A INPUT -s 99.27.202.67 -j DROP

iptables -A INPUT -s 99.31.123.191 -j DROP

iptables -A INPUT -s 99.36.16.156 -j DROP

iptables -A INPUT -s 99.38.203.228 -j DROP

iptables -A INPUT -s 99.4.136.182 -j DROP

iptables -A INPUT -s 99.58.203.81 -j DROP

iptables -A INPUT -s 99.68.231.37 -j DROP

iptables -A INPUT -s 99.69.161.123 -j DROP

Why bother going through this when I can just use sort inside of awk. The uglier and bloated way would be something like:

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP"}' filename | sort | uniq

Or even uglier:

curl www.infiltrated.net/blacklisted | ruby -ne 'puts $_ unless $_ == @prev; @prev = $_' | awk '{print "iptables -A INPUT -s "$1" -j DROP"}'

Comment by sil 178 weeks ago

This is exactly what I needed to see! Sil, you rock. The meticulous instruction was so helpful, particularly the placement of sort and its unique switch within awk versus (unnecessary) discreet use. I will use:

awk '!/#|[a-z]/&&/./{print "iptables -A INPUT -s "$1" -j DROP" | "sort -u"}'
Comment by surfkid 178 weeks ago

Excellent Contribution. Thank you.

Comment by LAN4N6 152 weeks and 5 days ago

Your point of view

You must be signed in to comment.

Related sites and podcasts