Welcome to WebmasterWorld Guest from 54.160.131.144

Forum Moderators: goodroi

Message Too Old, No Replies

Regarding robots.txt file

What does Disallow: /*? mean?

     

mt_biker

4:25 pm on Jan 12, 2011 (gmt 0)

10+ Year Member



I can't seem to find the answer to this one anywhere...

In this robots.txt file, does anyone know what the following Disallow line means:

User-agent: *
Disallow: /*?

Other than a Sitemap: line, this is the entire robots.txt.

mt_biker

9:42 pm on Jan 12, 2011 (gmt 0)

10+ Year Member



Just to clarify my initial question...

I get the Disallow: /* part (don't index anything), but what is the meaning of the question mark?

topr8

12:52 am on Jan 13, 2011 (gmt 0)

WebmasterWorld Senior Member topr8 is a WebmasterWorld Top Contributor of All Time 10+ Year Member




Disallow: /

means disallow the root and anything below it - effectively disallow all.

what they are saying is:
disallow anything that has a query string

eg
/page.php?a=b

mt_biker

3:54 pm on Jan 13, 2011 (gmt 0)

10+ Year Member



That makes sense. Thank you!

giggle

11:33 am on Jan 14, 2011 (gmt 0)

10+ Year Member



We have this:

User-agent: Googlebot
Disallow: /*?

What does it mean please? I tried reading through various pages but couldn't find the exact answer.

Thanks

aakk9999

1:08 pm on Jan 14, 2011 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



We have this:

User-agent: Googlebot
Disallow: /*?

What does it mean please? I tried reading through various pages but couldn't find the exact answer.


As said in the post above, this means:

what they are saying is:
disallow anything that has a query string

eg
/page.php?a=b

mt_biker

4:53 pm on Jan 14, 2011 (gmt 0)

10+ Year Member



User-agent: *

...means ALL bots whereas...

User-agent: Googlebot

...specifies the Googlebot only

coopster

5:42 pm on Jan 14, 2011 (gmt 0)

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



You may also want to review the forum "Charter" link at the top of this page. Each forum has a Charter with information pertinent to the topics and discussion on the forum. This particular forum has links and tutorials that answer many of your questions regarding the robots.txt directives.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month