Jump to content

Welcome, Guest!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

  • 0
Pedalman

robots.txt

Question

I just generated the robots.txt via Thirtybees backoffice and noticed that my old robots.txt had these lines at the top that are missing in the new one:

User-agent: *
Disallow:
User-agent: Googlebot
Disallow:
User-agent: Googlebot-image
Disallow: 
# Allow Directives

The new one looks like this:

User-agent: *
# Allow Directives

 

Should I bother? Do they mean the same or does the new one generated via Thirtybees block the google image bot?

Share this post


Link to post
Share on other sites

2 answers to this question

Recommended Posts

  • 0

This third 'Disallow:' in the old version looks like broken code, as there's nothing noted on what to disallow.

The other two directives disallow Google crawlers. Usually not what merchants want.

Share this post


Link to post
Share on other sites
  • 0

Thanks, I will have a look at Google Webmaster. Two years ago a SEO expert told me to add these lines ?! 😞

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...