Welcome to WebmasterWorld Guest from 54.197.116.116

Forum Moderators: goodroi

Message Too Old, No Replies

Regarding robots.txt file

What does Disallow: /*? mean?

   
4:25 pm on Jan 12, 2011 (gmt 0)

10+ Year Member



I can't seem to find the answer to this one anywhere...

In this robots.txt file, does anyone know what the following Disallow line means:

User-agent: *
Disallow: /*?

Other than a Sitemap: line, this is the entire robots.txt.
9:42 pm on Jan 12, 2011 (gmt 0)

10+ Year Member



Just to clarify my initial question...

I get the Disallow: /* part (don't index anything), but what is the meaning of the question mark?
12:52 am on Jan 13, 2011 (gmt 0)

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member




Disallow: /

means disallow the root and anything below it - effectively disallow all.

what they are saying is:
disallow anything that has a query string

eg
/page.php?a=b
3:54 pm on Jan 13, 2011 (gmt 0)

10+ Year Member



That makes sense. Thank you!
11:33 am on Jan 14, 2011 (gmt 0)

10+ Year Member



We have this:

User-agent: Googlebot
Disallow: /*?

What does it mean please? I tried reading through various pages but couldn't find the exact answer.

Thanks
1:08 pm on Jan 14, 2011 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



We have this:

User-agent: Googlebot
Disallow: /*?

What does it mean please? I tried reading through various pages but couldn't find the exact answer.


As said in the post above, this means:

what they are saying is:
disallow anything that has a query string

eg
/page.php?a=b
4:53 pm on Jan 14, 2011 (gmt 0)

10+ Year Member



User-agent: *

...means ALL bots whereas...

User-agent: Googlebot

...specifies the Googlebot only
5:42 pm on Jan 14, 2011 (gmt 0)

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



You may also want to review the forum "Charter" link at the top of this page. Each forum has a Charter with information pertinent to the topics and discussion on the forum. This particular forum has links and tutorials that answer many of your questions regarding the robots.txt directives.