Some fun with Blackhole for Bad Bots module

[cprats]

Blackhole for Bad Bots is a free module by @datakick very useful to block disrespectful crawlers. The system is very simple and effective: the module traps and blocks any originating IP for any bot crawling the directory /blackhole/, that has a disallow rule for all agents in robots.txt file.

But using .htaccess this module can also be very helpful to block a very large sort of nefandous hits by spammers, scanners, scammers, etc, etc, etc. I recently had an issue with spammer bots generating fake shopping carts, and Blackhole for Bad Bots module helped to stop that flood of garbage. All bots entered the site hitting a non existent folder, and they all followed this pattern. The result, after some code in .htaccess to redirect such visits to the black hole, was the complete annihilation of such annoyance.

Here is another example of what Blackhole for Bad Bots module can do for you, beyond of trapping nasty crawlers: for sure you get dozens of hits to non existent wp-login.php file, or to xmlrpc.php.  You should host your thirtybees installation alone in a server and not to mix it with other websites running WordPress (this is why you should do this). Here is a way to have fun with WordPress' brute force bots hitting thirtybees.

Place this code in your .htaccess file:

Redirect 302 /wp-login.php https://www.your-domain-name.com/modules/blackholebots/blackhole/
Redirect 302 /wp-config.php https://www.your-domain-name.com/modules/blackholebots/blackhole/
Redirect 302 /xmlrpc.php https://www.your-domain-name.com/modules/blackholebots/blackhole/

I've just redirected the hits that return a page-not-found.html that spends 33887 bytes. After the redirection to the black hole, the size of the file served is just 243 bytes, as the bot gets the module's ban warning, which is text only. So this will save you bandwidth and it will also block the source IP for that attack.

See in your server logs what your needs are, and modify the redirections at your convenience.

Of course, needless to say, never redirect to the black hole a folder or a file that effectively is in your server. If you have to block it, it is better to delete it. Redirect only bind hits to non existent files/folders by probe bots that only make you waste bandwidth.

[colorful-ant]

thanks - this looks very good - I'll try that tomorrow

[Scully]

Hello Datakick,

I recently installed the black hole module. It works during testing but it haven't trapped any bot so far. I am just guessing: Might it bee that the robots learn quickly about traps, expecially if the trap URI is fix?

And then: might it be an option to generate a random URI? I though I am familiar with PHP I couln't make out where the /blackhole/ URI comes from. Is it the directory within the module path or is it in the code? Yes I found the "getTrapUrl" function but changing code there didn't work as expected.

Best regards and thank you for your work.
Scully

[Scully]

I feel sorry to have learned lately that Datakick quit the TB project.

[Wartin]

On 6/9/2020 at 10:11 AM, Scully said:

And then: might it be an option to generate a random URI? I though I am familiar with PHP I couln't make out where the /blackhole/ URI comes from. Is it the directory within the module path or is it in the code? Yes I found the "getTrapUrl" function but changing code there didn't work as expected.

Hi, Scully. I found this great module and try it, as I'm having bots creating accounts and crawling for forms.

I managed to change the link of the blackhole, as I don't have the shop in '/'

Edited the file /modules/blackholebots/classes/blackholebots.php

  private function getTrapUrl() {
    $prefix = (int)Configuration::get('PS_REWRITING_SETTINGS') ? '' : rtrim($this->_path, '/');
    return $prefix . '/shop/blackhole/';
  }

Now every shop's pages have a '/shop/blackhole' link, and pressing there made them fall into THE TRAP! ha ha.

 

Thanks @datakick, great module.

[Wartin]

Hi again.

Reading logs, I could see this account-creating bot is accessing the very first link in the shop, this is, 'my account'. From there on, they can create a fake account.

Would it be possible to write the 'blackhole' link at the very start of the page? wouldn't it be better that in the end?

[the.rampage.rado]

@cprats, very nice idea!

I have plenty of those requests "//2019/wp-includes/wlwmanifest.xml" what should be the formatting in the first part of the redirect?

I'm using this generator https://redirectgenerator.netmagnet.cz/ and it works quite well for all other 404s but for those it returns different code that is not working with my server.

[DRMasterChief]

Wir haben mehr als 1 Domain in einem Webhosting aktiv, wird das in der Regel auch funktionieren oder müssen wir für jede Domain umleiten?

Redirect 302 /wp-login.php /modules/blackholebots/blackhole/
Redirect 302 /wp-config.php /modules/blackholebots/blackhole/
Redirect 302 /xmlrpc.php /modules/blackholebots/blackhole/

[the.rampage.rado]

It will work, yes.

[the.rampage.rado]

On 3/23/2024 at 3:42 AM, the.rampage.rado said:

I have plenty of those requests "//2019/wp-includes/wlwmanifest.xml" what should be the formatting in the first part of the redirect?

To update on a fix I found and worked for me.

All of those requests starting with // contain wp-includes, wp-admin or wp-content so this solves the issue:

 

RedirectMatch 301 wp-includes /modules/blackholebots/blackhole/
RedirectMatch 301 wp-admin /modules/blackholebots/blackhole/
RedirectMatch 301 wp-content /modules/blackholebots/blackhole/

 

[DRMasterChief]

Thanks @the.rampage.rado for this,  great idea 🙂

A question about the upper thing:

Redirect 302 /wp-login.php /modules/blackholebots/blackhole/
Redirect 302 /wp-config.php /modules/blackholebots/blackhole/
Redirect 302 /xmlrpc.php /modules/blackholebots/blackhole/

>> wouldn't it be better to use a Redirect 301  for this  (instead of 302)?

[the.rampage.rado]

Probably yes, but for sending them to the hole I doubt there's a difference, those bots probably don't care much about that.

[DRMasterChief]

I use the module on tb 1.5.1 and it has hits every day. I get the message from the module about it by email. But I also get two emails for each hit with the information that there are new warnings in the log files. This is at the same time as the message from the module. Why is that?

[the.rampage.rado]

Could you post those logs (delete any sensitive info first)?

[DRMasterChief]

this is one of the questions about this...   where can i find these logs?

[Petter]

I'm using Nginx and Nginx do not use .htaccess file. Can't the same be done using fail2ban ?

[Yabber]

On 6/28/2024 at 6:31 PM, Petter said:

I'm using Nginx and Nginx do not use .htaccess file. Can't the same be done using fail2ban ?

Install Lua module to nginx, you will have nginx with htaccess support: https://github.com/e404/htaccess-for-nginx?tab=readme-ov-file