Bad Behavior 2
July 4th, 2006 by Michael Hampton
It’s been a long time coming, and Bad Behavior 2, the next generation of the Web’s premier malicious traffic killer, is finally here!
Bad Behavior, conceived in 2005 as a fingerprinting method for HTTP requests, has proven, as one user called it, “shockingly effective” at identifying and blocking malicious activity, including blog/wiki spam, e-mail address harvesting, automated cracking attempts, and more. It does all of this looking only at the HTTP request headers; for POST data, the content of the spam is not analyzed at all.
Even so, Bad Behavior blocks the vast majority of web spam, and has gotten the spammers so worked up they’ve actually stopped spamming me with their latest tools, so as to try to prevent me from learning what they’re up to. (It didn’t work. “The king hath note of all that they intend, By interception which they dream not of.” — Shakespeare)
I’ve been developing Bad Behavior 2 in my limited spare time, off and on, for almost a year. And I want to thank all of you for your patience, especially while spammers were bombarding your blogs and wikis, and for your support. It’s been a crazy year, and I’ll be talking more on a personal note about it in the next few weeks.
And that is the reason I am releasing the software now, when not all of the planned features are present: In recent weeks spammers have greatly stepped up their activity, with some sites receiving ten times as much spam as before. I’ve been hard at work on Bad Behavior 2, making sure that it can block this spam without keeping away your regular readers.
New Features
Even without everything I’d planned, Bad Behavior 2 is chock full of new features. Some of them are quite visible, others are more in the backend.
- Bad Behavior 2 is faster than Bad Behavior 1, whether you use database logging or not. It has been completely redesigned from the ground up to be as fast as possible and provide protection on very high traffic sites, such as when you find yourself on the front page of slashdot.org, or you’re the sysop of Wikipedia. For most requests, Bad Behavior 2 issues at most one fast database query, and in many cases, no database queries. Bad Behavior’s run time on fast servers is measured in single milliseconds.
- Bad Behavior 2 has been enhanced with additional checks for spammers who have started or increased their activity in the last year. It also has better screening of trackback spam, killing virtually all of it. Bad Behavior 1 permitted a lot of trackback spam.
- Bad Behavior 2′s options have been standardized across ports, so that the same options work the same way on each software package. (Not all of the options apply to each package, however.) This makes Bad Behavior easier to deploy across multiple sites and different software packages.
- On some software packages, Bad Behavior’s options can be controlled from within the software package. Currently an administrative screen is available on WordPress, and a screen is planned for MediaWiki. (It hasn’t been implemented because developer documentation is sparse, incomplete and wrong, according to Brion. When the documentation improves, the MediaWiki port’s features will improve.)
- For speed reasons, Bad Behavior 2 does not use PHP classes in its core. But Bad Behavior 2′s API has been rewritten to provide a better interface for certain types of software, such as ExpressionEngine, which expect their extensions to be encapsulated in classes. (The EE port isn’t complete, sorry!)
- Some spam delivery methods are easily confused with legitimate users, especially those in large corporations or governments. This is mainly due to the proxies in use at those places. When a spammer uses such a proxy, Bad Behavior cannot easily tell whether the request is legitimate or not. In Bad Behavior 1, these requests were blocked, causing many legitimate users to be blocked. In Bad Behavior 2, you can choose whether to block these requests with the “strict” option.
Upgrading
To upgrade to Bad Behavior 2, you first need to remove all previous versions of Bad Behavior, including any 2.0 pre-release versions. Then you need to drop any database tables Bad Behavior may have created in your database. These may be named, e.g. mw1_bad_behavior or wp_bad_behavior. They may also be bad_behavior_log instead.
Then you are ready to install Bad Behavior 2!
Installation
The basic installation instructions haven’t changed much from Bad Behavior 1. Please see:
Options
For all platforms except WordPress (for now) options are configured by editing them near the top of the bad-behavior-platform.php file. Currently this includes MediaWiki and the generic non-database port. MediaWiki options will be moved to a special page in a future version.
In WordPress, the available options appear in the Options » Bad Behavior administrative page.
The options available to all users are:
- log_table: The name of the database table Bad Behavior should use. This is set by default for all platforms and should not be changed unless you are porting Bad Behavior to a new software package.
- display_stats: When this option is set, Bad Behavior will display statistics in the footer of your web pages. (Currently works only on WordPress.)
- strict: Enables strict mode blocking. When turned on, certain types of spam will be blocked, but legitimate corporate and government users may also be blocked. This is off by default.
- verbose: Enables logging of all requests received. When turned on, the details of every HTTP request Bad Behavior processes will be logged to the database. When turned off, only blocked requests, and a few legitimate but suspicious requests, will be logged. This is off by default.
To-Do List
I’ve pushed this release out the door because it’s proven stable, fast, and effective, and because spammers have greatly stepped up their activity. So several features which were in the roadmap have been postponed. I will be drawing up a new post-2.0 roadmap for these features in the next few days.
Finally…
As always, if you find Bad Behavior valuable, please consider making a financial contribution. I develop Bad Behavior in my spare time, and every little bit means I have more spare time to devote to its development.
And don’t forget to subscribe to the RSS feed or the mailing list. (They’re the same content.)
war59312 Says
Awesome!
Working great, as usual!
Jul 5th, 2006 at 3:44 am
Paul Burdick Says
In your generic file for Bad Behavior you are still using $wgDBprefix, which is obviously not a variable set in other systems. I am working on getting a test Extension set up for ExpressionEngine now.
Jul 5th, 2006 at 8:40 pm
Michael Hampton Says
Oops! Well, it doesn’t matter too much what’s in there; I think it’s commented out anyway.
Somewhere I’ve got a skeleton EE extension; just haven’t had time to look at it (and reinstall EE on a testbed). I’ll dig it out sometime in the next few days, though I suspect whatever you come up with is going to be a lot better than the complete garbage I write.
Jul 6th, 2006 at 1:55 am
Viper007Bond Says
Hmm, I can’t get my stats to show up in my footer. Any ideas why?
Jul 6th, 2006 at 7:32 am
Gary Says
Cool. Can’t wait to try this baby out.
Jul 6th, 2006 at 10:05 am
Hal Rottenberg Says
Yay for strict mode! Got it installed on my MW, hopefully the new stuff you’ve got in here will keep out the latest nasties.
Jul 6th, 2006 at 1:58 pm
Michael Hampton Says
Hal, it should pretty neatly solve your particular spam problem.
Jul 6th, 2006 at 2:14 pm
Hal Rottenberg Says
Michael, guess what, more spam. I tried to email you, it failed again. I’ve uploaded the sql to http://halr9000.com/stuff/mw_bad_behavior.sql.gz
Jul 7th, 2006 at 2:53 pm
Diwaker Says
I’m having a little problem with BB2 (infact, ever since the beta I’ve been having this problem). I use the Javascript tabber from [1] on my website [2]. When BB2 is enabled, the script doesn’t seem to be able to run (i.e., I only see the “fallback” HTML code, not the tabs). However, I see this problem only on Firefox, on Konqueror everything seems to be fine. Things work just fine on all browsers with BB1. Any ideas? (I’m going to leave BB2 on for a couple of hours in case you want to take a look — after that I’ll disable it, I want my tabs back on Firefox!). Thanks
[1]
[2]
Jul 8th, 2006 at 4:19 pm
Diwaker Says
The previous comment didn’t let the URLs pass through for some reason. Here’s another attempt:
ONE: http://www.barelyfitz.com/projects/tabber/
TWO: http://floatingsun.net/blog/
Jul 8th, 2006 at 4:21 pm
Michael Hampton Says
I pulled up your web site, but I could not find any evidence of that other JavaScript code. What are you talking about?
Jul 8th, 2006 at 5:01 pm
BillSaysThis Says
Michael, there’s no mention of BB Blackhole here. Can you add a sentence or two on its status for those of us using it with BB 1.x?
Jul 8th, 2006 at 5:19 pm
Michael Hampton Says
Okay. Bad Behavior Blackhole is still running, though it’s hardly been looked at in a year or more. I plan to resurrect it in the near future, when I get some time. (And time is money, hint hint…)
Jul 8th, 2006 at 5:28 pm
Sara Says
Thank you so much for updating to version two!
I installed this an hour after you posted the download. So far the plugin has blocked 103 attempts at the time of this writing. I checked the database table, and sure enough, there were practically waves upon waves of spam that akismet would of had to process, but never touched due to the fact that BB2 stopped it first. Now I know why my site has been running a bit slow last night…I got attacked with around 50 spam comments and all of them were caught by BB2.
If I had some extra cash, I would be happy to donate, but all I can offer is maybe perhaps an artistic service, heh.
Thank you so much for the time and effort you put into this.
(BTW, I am running WordPress 2.0.3]
Jul 8th, 2006 at 5:41 pm
TechZ Says
I love it, it’s sooo easy to use! I use Spam Karma 2 already, and now this, I’m quite safe
Thanks!
Jul 8th, 2006 at 6:20 pm
Craig Hartel Says
Michael,
Within 30 seconds of installing this latest version BB stopped a spammer from getting through. I can’t begin to tell you how much I appreciate all of the work that you have put into BB. I will certainly be making a donation and I sincerely encourage everyone who uses BB to send you a few dollars..Bad-Behaviour is a bargain at any price!
Jul 8th, 2006 at 6:33 pm
Computer Guru Says
Are you blocking the entire RIPE network?
I can’t download the plug-in. (Error 400) RIPE is _all_ over Europe…. millions of legit users.
Jul 12th, 2006 at 7:34 am
Michael Hampton Says
Computer Guru: What are you talking about? Try READING the page which came up when you were blocked. It’s clear you can speak English, so I hope that’s not going to be a problem for you.
Jul 12th, 2006 at 1:46 pm
Tarun Says
Any word on if this will be coming to IPB forums soon? I’ve been seeing a lot of IPB forums getting hit with spam pretty bad.
Jul 12th, 2006 at 2:36 pm
Tarun Says
Sorry, I omitted one thing. I’ve worked with a webmaster to try and get Bad Behavior 2 working on his site and forums, the problem is that whenever someone uses the reply or add post/topic buttons, it sends them to the index. This happens when it’s set to use the generic BB2 from the functions.php file.
Jul 12th, 2006 at 2:39 pm
Thomas Says
Sorry, but I don’t know where to ask and it seems that the “gods” are here!
Anyone knows what this one is?
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; SIMBAR Enabled; .NET CLR 1.1.4322)
Seems to be a new one?! This one tried to paste Spam into one of our forums.
Kind regards, Thomas
Jul 17th, 2006 at 4:15 pm
Michael Hampton Says
The User-Agent is pretty much irrelevant. If you can send me a bad_behavior log entry, I can look into it further.
Jul 17th, 2006 at 4:21 pm
J Says
The current version 2.0.3 is the BEST release ever! Thanks Dude. I have not a single spammer coming through since weeks! Great job man! Thanks! J.
Jul 27th, 2006 at 2:59 pm
CoralSea Says
I’ve been using Bad Behavior for months now with great results. I run it in generic mode to block spam from a guestbook. I installed 2.0.5 about a week ago and now some spammers are getting through
Has anyone ported generic to work with MySQL/Apache? I know enough php/mysql to be dangerous but can’t quite figure it out.
Aug 11th, 2006 at 2:59 pm
eyn Says
I think you missed some error in your latest BB 2.0.5. In bad-behavior-generic.php line 84 you put in an extra ” in the return statement.
Aug 17th, 2006 at 10:30 am