Spam shopping carts

[SLiCK_303]

the rest of robots.txt has it like: Disallow: */classes/ do we do it like that, with the *, or like you said: Disallow: /blackhole/

[movieseals]

@slick_303 On my version, I did this: Disallow: */blackhole/ Seems to work.

[SLiCK_303]

@movieseals thats what i did too, since the rest of the disallows were that way

[movieseals]

@datakick Very nice implementation! Great addition to the TB arsenal!

[datakick]

@SLiCK_303 @movieseals - robots.txt standard technically does not recognize asterisk as a wildchar. So, if some bot implemented robots.txt handling strictly according to specification, they wouldn't understand your directive and could actually follow the link.

Google, Bing, and other major players support this kind of extension to the standard, so they will process it properly. But I think it's safer to use Disallow: /blackhole/, or even Disallow: /blackhole to be sure no bot with good intentions will be caught in this trap.

Basically, this directive means that any url starting with /blackhole/ is prohibited from browsing. Your (nonstandard) directive says that any url containing /blackhole/ is prohibited.

[movieseals]

@datakick said in Spam shopping carts:

Disallow: /blackhole/

Thank you for the clarification. Makes sense!

[datakick]

just FYI, I've added this module to the store. I'm not going to create a separate forum thread for this module, but if anyone has feature requests feel free to create one