BlackHole for badbots emails

[haylau]

Is it possible to stop the emails about badbots?

[AndyC]

I'm still waiting to find out what they all mean from a post from last year I think.. If there is no way to understand them I would also like to stop getting them

[datakick]

Of course you can disable the email. There are many ways to do this:

1. disable / uninstall module
2. edit module and comment Mail::Send command
3. delete email template
4. install conseqs module and set up rule to block this particular email
5. and more...

I personally think it's not very good idea to do this, though. You should always verify that the trap didn't catch some valid bots / users. It shouldn't be such a big deal - with properly set up robots.txt, you should catch only a few bots per month. I personally receive the email maybe once every two months

[DRMasterChief]

We have also been using the module for a few days. Approximately every day 1-2 reports arrive via email about a captured bot.

I would also like to understand the e-mails better and therefore i am asking for information here.

We get the message something like this:

Bad Bot Alert!
IP Address: 4.2x5.1x0.15

      WHOIS Lookup for 4.2x5.1x0.15
      ARIN WHOIS data and services are subject to the Terms of Use
available at: https://www.arin.net/resources/registry/whois/tou/
If you see inaccuracies in the results, please report at
https://www.arin.net/resources/registry/whois/inaccuracy_reporting/
Copyright 1997-2024, American Registry for Internet Numbers, Ltd.

NetRange:       4.240.0.0 - 4.255.255.255
CIDR:           4.240.0.0/12
NetName:        MSFT
NetHandle:      NET-4-240-0-0-2
Parent:         NET4 (NET-4-0-0-0-0)
NetType:        Direct Allocation
OriginAS:       
Organization:   Microsoft Corporation (MSFT)
RegDate:        2021-04-06
Updated:        2021-04-06
Ref:           https://rdap.arin.net/registry/ip/4.240.0.0
OrgName:        Microsoft Corporation
........

Do I now have to look for who the IP belongs to, for example whether it is from the search engine Bing in this case (Microsoft)? That would be important to index the shop...

And:    can I change the email address to which notifications are sent? Then we could use a separate address or mailbox for this.

Edited May 8 by DRMasterChief

[the.rampage.rado]

If it says Microsoft it does not mean it's a Bing or MS service. It might be a machine hosted on their hosting. The same thing you can see with AWS or other known hosting companies.

In any case the vast majority of bots you will catch will be from northern China provinces where they host their hacking.

[DRMasterChief]

Hi, not one Chinese yet,  so this is why i have these questions.....   i have IP´s from MS, OVH, Ionos....  and i am not sure if they are 'real' the Bot's we want to catch.

[datakick]

11 minutes ago, DRMasterChief said:

Hi, not one Chinese yet,  so this is why i have these questions.....   i have IP´s from MS, OVH, Ionos....  and i am not sure if they are 'real' the Bot's we want to catch.

Did you correctly set the robots.txt when you installed the module ?

[DRMasterChief]

i guess so:

# robots.txt automatically generated by thirty bees e-commerce open-source solution
# http://www.thirtybees.com - http://www.thirtybees.com/forums
# This file is to prevent the crawling and indexing of certain parts
# of your site by web crawlers and spiders run by sites like Yahoo!
# and Google. By telling these "robots" where not to go on your site,
# you save bandwidth and server resources.
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/robotstxt.html
User-agent: *
# Allow Directives
Allow: */modules/*.css
Allow: */modules/*.js
# Private pages
Disallow: /blackhole/
Disallow: /modules/blackholebots/blackhole/
Disallow: /*?orderby=
Disallow: /*?orderway=
Disallow: /*?tag=
Disallow: /*?id_currency=
Disallow: /*?search_query=
Disallow: /*?back=
Disallow: /*?n=
Disallow: /*&orderby=
Disallow: /*&orderway=
Disallow: /*&tag=
Disallow: /*&id_currency=
Disallow: /*&search_query=
Disallow: /*&back=
Disallow: /*&n=
Disallow: /*controller=addresses
Disallow: /*controller=address
Disallow: /*controller=authentication
Disallow: /*controller=cart
Disallow: /*controller=discount
Disallow: /*controller=footer
Disallow: /*controller=get-file
Disallow: /*controller=header
Disallow: /*controller=history
Disallow: /*controller=identity
Disallow: /*controller=images.inc
Disallow: /*controller=init
Disallow: /*controller=my-account
Disallow: /*controller=order
Disallow: /*controller=order-opc
Disallow: /*controller=order-slip
Disallow: /*controller=order-detail
Disallow: /*controller=order-follow
Disallow: /*controller=order-return
Disallow: /*controller=order-confirmation
Disallow: /*controller=pagination
Disallow: /*controller=password
Disallow: /*controller=pdf-invoice
Disallow: /*controller=pdf-order-return
Disallow: /*controller=pdf-order-slip
Disallow: /*controller=product-sort
Disallow: /*controller=search
Disallow: /*controller=statistics
Disallow: /*controller=attachment
Disallow: /*controller=guest-tracking
Disallow: /*controller=trigger
# Directories
Disallow: */classes/
Disallow: */config/
Disallow: */download/
Disallow: */mails/
Disallow: */translations/
Disallow: */tools/
# Files
Disallow: /de/adresse
Disallow: /de/adressen
Disallow: /de/authentifizierung
Disallow: /de/warenkorb
Disallow: /de/discount
Disallow: /de/auftragsverfolgung-gast
Disallow: /de/bestellungsverlauf
Disallow: /de/kennung
Disallow: /de/mein-Konto
Disallow: /de/bestellung
Disallow: /de/bestellbestatigung
Disallow: /de/bestellungsverfolgung
Disallow: /de/schnell-bestellung
Disallow: /de/bestellschein
Disallow: /de/kennwort-wiederherstellung
Disallow: /de/suche
Disallow: /en/adresse
Disallow: /en/adressen
Disallow: /en/authentifizierung
Disallow: /en/warenkorb
Disallow: /en/discount
Disallow: /en/auftragsverfolgung-gast
Disallow: /en/bestellungsverlauf
Disallow: /en/kennung
Disallow: /en/mein-Konto
Disallow: /en/bestellung
Disallow: /en/bestellbestatigung
Disallow: /en/bestellungsverfolgung
Disallow: /en/schnell-bestellung
Disallow: /en/bestellschein
Disallow: /en/kennwort-wiederherstellung
Disallow: /en/suche