Crawler fills 100+ cart and leave them abandoned?

[DRMasterChief]

Hello, as we will go live with our TB shop in summer i think about the following, which seems to be a problem in a lot of Prestashops:

A lot of PS users gets visits from Seokicks, who put everything into the shopping cart, and then leave them abandoned. This needs technical resources, slow down the shop and should be avoided (also other crawlers, i think there are a lot of different out there). I know from PS users which have 4000+ filled carts in a about 1-2 weeks.

I am not sure if this could be solved by an .htaccess (maybe the most unfavorable solution) or by blocking IP addresses? But IP addresses can change very fast and i think it is not possible to be up-to-date with this and the crawlers.

Does anyone have experiences with this and TB? Is this the case also at TB shops?

Can we do this with an script in the .htaccess like the following ?? SetEnvIfNoCase user-agent SEOkicks spam_ref = 1 .....same for other crawerls

I think we should think about that and also if this appears with TB, too. Hopefully the developers can contribute some ideas and solutions. thank you!

[datakick]

I think it's really waste of your resources to battle this. You just can't stop these bots in generic way - not without significant risk of impacting your regular customers. And to try to fix each and every one of them is another extreme. Your time is much more valuable than few thousands rows in database these bots generates, or the bandwidh they consume.

I would say that the best defense is to simply delete abandoned carts regularly. For annonymous visitors delete any cart that's older than few days. You will probably want to keep customer's carts a little big longer, say for few weeks, or even the months.

[DRMasterChief]

hmmm but 40k abandoned carts in 9 months are a lot :( (also the delete them manually) Please do not underestimate this.

[datakick]

@DRMasterChief I never said to handle this manually, that's even bigger waste of time. There are modules that can delete them automatically, you just need to setup cron task. With proper retention policy there will never be more than few hundred abandoned carts...

[datakick]

for example, this module looks promising and 30bz compliant

[movieseals]

I have the same bot problem and I use tidy, the module datakick talks about, and it does the cart cleaning as well as a host of other very useful tasks. Definitely worth it!

[DRMasterChief]

But wouldnt it be much better to keep the bots out, and not let them search and fill anything first? That would be my preferred solution. I also found a lot of more here: https://www.askapache.com/htaccess/setenvif/

[movieseals]

I have found this solution, which is free, and works with most bots: https://perishablepress.com/blackhole-bad-bots/

It has considerably curbed the problem for us. It is really to implement in TB, not as much on Prestashop but it is feasible.

Good luck!

[DRMasterChief]

@movieseals Ah really, do you use this for TB? I thought it is for Wordpress only?

[movieseals]

The plugin does not work with TB, but the PHP script version can be implemented anywhere. The instructions are pretty straight forward.

[BlindPenguin]

Looks like i'm a bit late, but for anyone interested:

I also had this problem on a regular basis on my old shop. Bots usually follow links. If you get abandoned carts, it means that somewhere in your theme the "Add to Cart" buttons are defined as a link. Most of the time it's the list view on the front page or category pages. To prevent them from following those links, you should change them to buttons and call them via javascript. Just like in the product view page in the default theme.

Alternatively you could remove those buttons from list view completely. I don't think people would add anything to cart from a list view. Well, i wouldn't.. don't know about anyone else.

[led24ee]

I have still this abandoned cart problem. Some hundred in one day. Is there any solution for this ? Is there any simple solution to temporary ban  IP witch only purpose is just make shit in shop ?

[datakick]

Have you tried blackhole for bad bots module? Might help. If you know the IP address upfront, you can also pre-populate the blacklist:

INSERT INTO tb_blackholebots_blacklist(address, date_add, date_upd, cnt)
VALUES('123.123.123.123', now(), now(), 1);

 

[led24ee]

These are random IP-s. With more anonymous server this problem only grows. Many big platform have similar problems and there are restrictions like temporary ban or time delay between operations.

[oky]

On 7/12/2022 at 2:55 AM, led24ee said:

I have still this abandoned cart problem. Some hundred in one day. Is there any solution for this ? Is there any simple solution to temporary ban  IP witch only purpose is just make shit in shop ?

there is a module Advanced anti-spam and fake accounts + Re-Captcha

[led24ee]

Another module for leasing ? Thanks, but no, thanks.