Jump to content
thirty bees forum

Google is crawling bogus pages


netamismb

Recommended Posts

In Google Console, Google is crawling a lot of pages (more than 1k pages)  with index.php?controller=trigger&ts=1710225603 which is going to {"status":"failed","error":"Forbidden"}. 

Any idea how I can get rid of them? 

Link to comment
Share on other sites

  • 4 months later...

Search-engines are asking for sitemaps, but push into the index everything, even pages the engine create.
Bing e.g has pages in the index which were outdated in 2019. As DDG took over their indexes so those URLS are in there too.
So in love with AI that they forgot all about cleaning their indexes.

Link to comment
Share on other sites

15 hours ago, the.rampage.rado said:

Regarding the discovered pages in the Search Console - I'm curious if they can be removed or should we leave them as is?

You can't remove them. They should go away over time. But I have entries in search console, that were fixed a year ago (with canonical and so) and google still tries to crawl the old version. The search console even tells, that there is no referring page to the url they want to crawl 😂 🤦‍♂️

It surely helps, when you have the bleeding edge AND the robots.txt disallow.

Personally I think this has no realy influence on SEO, but it helps to clean search console. With a clean search console it's simpler to find real issues...

Edited by wakabayashi
Link to comment
Share on other sites

16 hours ago, DRMasterChief said:

it is closed at Github right now,  can anyone tell the solution here?  thanks 

The solution is to demand that google fix this bug on their side.

If your robots.txt instructs google to ignore some url, then google console  should not display warning that the url is not crawlable.

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...