Category Archives: Uncategorized

I had to temporarily block all user registrations on the bug tracker for a short while due, ironically, to spam registrations.

Registration has been reopened for now, and I’m manually dealing with the spammers.

This very bad experience has motivated me to do more about web spammers.

In this case, the bug tracker is Redmine, a web application developed in Ruby on Rails.

My first thought in such a case would have been to throw Bad Behavior at the web app and let it handle the problem. Unfortunately, Bad Behavior is written in PHP, making this impossible. By this point there must be any number of Ruby/Rails apps out there which are now suffering from the spam problems that we all went through and mostly got under control years ago.

It’s therefore my intention, as part of the Bad Behavior 3.0 rewrite, to create a Ruby gem which can be used to help secure such web applications. It may not be a simultaneous release, but something has to be done, and soon.

P.S. It’s also come to my attention that Redmine hasn’t been emailing me when someone enters a new ticket into the system. This should also be fixed. I’ve also taken the time to look at every ticket currently in the system. All bugs and support requests have been responded to, and all features looked at.

It’s come to my attention that Google has once again managed to get FeedBurner blacklisted at Project Honey Pot.

Those of you using Bad Behaavior’s Project Honey Pot support will see that requests from FeedBurner are blocked because the IP address is on the http:BL blacklist.

This is caused by an architectural problem at Google, and will require Google to resolve the issue for the problem to go away permanently. The issue is that, in the case of FeedBurner, Google uses IP addresses which are shared by third parties using Google App Engine, some of which are spammers. The spammers quickly get Google’s IP address blacklisted all over the Internet, and suddenly FeedBurner stops working.

If you are impacted by this issue, you can whitelist the affected IP addresses or the FeedBurner user agent string, or disable Project Honey Pot. Be aware that doing any of these will increase the amount of spam you receive. You should also complain to Google, since this isn’t the first time this has happened, and they seem to have done absolutely nothing about it.

Bad Behavior 2.2.13 has been released. This is a maintenance release and is recommended for all users.

Changes

The following changes have been made since 2.2.12:

• Requests from the Baidu search engine now go through screening similar to Google and other major search engines. This will help to prevent illegitimate access from clients which falsely claim to be the Baidu search engine. A logic error which prevented these checks from ever running has been fixed.

Download

Download Bad Behavior now.

Notes

Just as a reminder, if you use CloudFlare on your site, you must enable the Reverse Proxy option in Bad Behavior’s settings, or many of your visitors and search engines will be blocked.

Work on Bad Behavior 3.0 is finally making progress; I have some very basic almost-functional pre-alpha code and I hope to have it cleaned up enough to attempt to begin using in the next few weeks. Since this is usually the slow season for me, I hope to have some extra time to work on it over the holiday season. If you would like to see this work progress more quickly, or you just want to say thank you, consider making a donation today.

Bad Behavior 2.2.12 has been released. This is a maintenance release and is recommended for all users.

Changes

The following changes have been made since 2.2.11:

• Search engine screening by IP address is now more lenient; a failure to match a known IP address range no longer blocks the bot outright. This change is in response to a major search engine which is adding large numbers of IP address ranges faster than they can be tracked and added to Bad Behavior. Requests which don’t match a known IP address range still go through normal screening, while requests which match will be passed immediately.
• Search engine IP address screening is bypassed when the request originates from an IPv6 address, pending the addition of IPv6 subnet matching code.
• Requests from the Baidu search engine now go through screening similar to Google and other major search engines. This will help to prevent illegitimate access from clients which falsely claim to be the Baidu search engine.
• Some URL blacklist strings have been removed due to the possibility of their matching legitimate user input (e.g. in a site search phrase).

Download

Download Bad Behavior now.

Notes

Just as a reminder, if you use CloudFlare on your site, you must enable the Reverse Proxy option in Bad Behavior’s settings, or many of your visitors and search engines will be blocked.

Work on Bad Behavior 3.0 is finally making progress; I have some very basic almost-functional pre-alpha code and I hope to have it cleaned up enough to attempt to begin using in the next few weeks. Since this is usually the slow season for me, I hope to have some extra time to work on it over the holiday season. If you would like to see this work progress more quickly, or you just want to say thank you, consider making a donation today.

Oops! If you attempted to download Bad Behavior 2.2.11 before right now, when the announcement was posted to this site, you received a copy with a syntax error, which generated a PHP fatal error.

I’ve replaced the download with a corrected version. If you are experiencing this issue, you can remove and reinstall Bad Behavior, or edit line 94 of bad-behavior/blacklist.inc.php and add a , (comma) after the second " (quotation mark) so that the beginning of the line reads "Nikto",.

As I’ve noted previously, I’m in the midst of moving away from WordPress subversion and toward git (and github) which will let me put in place processes to prevent this sort of brown paper bag problem again. Please accept my apologies for the inconvenience this may have caused you.

Bad Behavior 2.2.11 and 2.0.49 have been released. This is a maintenance release and is recommended for all users.

NOTE: Support for the 2.0 series is very limited and will end June 30, 2013. Plan to migrate to the 2.2 series as soon as possible.

Changes

The following changes to 2.2 have been made since version 2.2.10:

• Google AdSense has changed their crawler’s User-Agent string to a string that matches a user agent blacklist entry. This would prevent the delivery of targeted ads to a page, and result in generic ads being displayed. The blacklist entry was temporarily removed pending communication with Google.
• A PHP warning would be generated if any whitelist had blank lines in it. Blank lines are now stripped out of whitelist entries.

The following changes to 2.0 have been made since version 2.0.48:

• Google AdSense has changed their crawler’s User-Agent string to a string that matches a user agent blacklist entry. This would prevent the delivery of targeted ads to a page, and result in generic ads being displayed. The blacklist entry was temporarily removed pending communication with Google.

Download

Download Bad Behavior now.

Bad Behavior 2.2.10 has been released. This is a maintenance release and is recommended for all users.

Changes

The following changes have been made since version 2.2.9:

• Code added in the previous release to support detection of malicious attacks contained an unfortunate typo causing PHP warnings to appear. This has been fixed.

Download

Download Bad Behavior now.

Bad Behavior 2.2.8 has been released. This is a maintenance release and is recommended for all users.

Changes

The following changes have been made since version 2.2.7:

• Several robots associated with spam and malicious activity have been identified and blocked.
• WordPress: A minor bug causing the whitelist to not appear properly in the administrative page in some circumstances has been fixed.

Download

Download Bad Behavior now.

Notes

While reviewing the site for the recent disaster recovery, I noted that some ports of Bad Behavior had not been updated in a very long time and do not use the new 2.2 code base, and some which appear to have been abandoned. These have been noted on the list of ports as “legacy” and “abandoned” respectively. If you are a port maintainer, or you think you may want to be, please check the list for your platform.

I also noted that some current ports were released under the GPL version 2 only. Since Bad Behavior 2.2 uses the LGPL version 3 (or any later version) the license is not compatible with GPLv2 only connectors. I’ll be contacting port maintainers individually about these to attempt to resolve these issues, but if you are one and you are aware of this, please update your license to GPLv3 or later, or LGPLv2.1 or later.

Finally, thank you to all of you who provided kind words, offers of technical assistance and of course donations during this very stressful disaster recovery. If you haven’t contributed lately, or at all, please help me keep Bad Behavior going by donating today.

You may have noticed that we’ve been offline for over a week now. This message is meant to explain what happened and what I’ve been doing to address the problem.

On June 1, the server on which my sites were hosted suffered hard drive failure and the filesystem became corrupted before the server could be pulled offline. Recovery is still in progress on the drive and it’s possible that I’ll be able to recover part or all of the server data, though that seems unlikely at this point.

It was at this point that I discovered that my off-site server backups were also corrupted and mostly unusable. Since then I have been recovering data from various other sources, such as very old offline backups, Google cache and even the Internet Archive. I have completed this process and have evaluated the recovered data.

While I was able to recover most of the old posts, I made the decision not to restore them. Nearly all of them contained outdated and useless information. I did restore most of the pages with the documentation, though, rearranging the information somewhat and making a few minor updates. Shortly I’ll be writing all new documentation for Bad Behavior anyway.

About a week ago I obtained a new server and have been configuring it to prepare it to host my sites going forward. The first thing installed was a versioned off-site backup system, from which it’s possible to roll back to different versions of each file (in case a corrupt file gets backed up, you can go back to the previous good copy). This will hopefully help prevent this situation from occurring in the future. (For the technically minded among you, I’m now using rdiff-backup, which is working well.)

The server is now fully configured and doing proper backups, and is now hosting the recovered web site.

I had to put virtually everything else on hold to complete this recovery, which affects several other sites than Bad Behavior and has absolutely destroyed my schedule and dramatically reduced my ability to make income, until the recovery is complete (and I still have a few other smaller sites to recover). If you’d like to support Bad Behavior during this critical time, please click here.

While we’re on the topic, if you have anything even remotely important on a computer, please do yourself a favor and ensure that you’re making backups – and that you can successfully restore from the backup! Eventually something like this will happen to you, and you will be glad you have the backups.

Finally, let me take a moment to thank you all for using Bad Behavior and for your support over the years. I actually have a new release ready, which blocks a couple of new spambots, and it will be posted in the next day or so. Again, if you’d like to support Bad Behavior during this time, please click here.

It seems the European Union has gone absolutely insane with extreme requirements for websites which use cookies. As Bad Behavior uses a cookie, and I’ve received requests for information about it, this post should serve to disclose (probably in much more detail than you really need) what Bad Behavior’s cookie is used for and what you may need to tell users to comply with the new EU regulations.

Bad Behavior sets a single session cookie named “bb2_screener_”. This cookie, which expires at the end of the user’s browser session, records the user’s IP address and time of their most recent visit to your site. The cookie is sent directly from your site, and not by a third party.

Bad Behavior uses this cookie to determine whether a request is a spammer who is rotating through different IP addresses. It is therefore used to maintain the security of your web site.

This cookie is solely used for the security of your web site and is not used for marketing purposes. For the purpose of site security, you may choose to share logs of user requests containing this cookie with a third party (me). If you share these records with me, I use them for the purpose of improving Bad Behavior’s detection of spam, and for no other purpose, I maintain them in encrypted storage, and I delete the records within 90 days.

Currently Bad Behavior has no means to determine whether your users have given consent to receive cookies. Since it is used solely to maintain the security of your site from malicious activity, you may be able to argue that it is exempt from the consent requirement, but since this requirement is new and untested in the courts, it’s not clear whether this will work. It’s also unclear at this time how to integrate such a requirement into Bad Behavior and all of the various possible platforms on which it can run.

If you feel you need any additional information in order to comply with these bizarre new requirements, just contact me and I’ll get you the information you need.

P.S. Bad Behavior is still in need of your support. If you haven’t donated recently, or at all, please consider doing so.