Forum Moderators: goodroi

Message Too Old, No Replies

Disallow: /#example.html#

How would the following be viewed by a spider?

         

kgormat

7:58 pm on Jun 23, 2006 (gmt 0)

10+ Year Member




How would the following be viewed by a robot?

Disallow: /#example.html#

Since the # sign often designates a note, would the spider view this

Disallow: /

Pfui

12:23 am on Jun 24, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Pardon me but what do you mean by "note"?

If you provided more specifics, please, it'd be easier to understand your Q and help you. You can use words like example and such to represent real text.

Little_G

1:41 am on Jun 24, 2006 (gmt 0)

10+ Year Member



Hi,

The # character in robots.txt [robotstxt.org] designates the start of a comment, so yes it should be ignored by a bot.

# robots.txt for http://www.example.com/

User-agent: *
Disallow: /cyberworld/map/ # This is an infinite virtual URL space
Disallow: /tmp/ # these will soon disappear
Disallow: /foo.html

Andrew

kgormat

3:29 pm on Jun 24, 2006 (gmt 0)

10+ Year Member



Thanks for the resource Little_G. Thats exactly what I was looking for.

Pfui, I actually meant to say 'comment', not 'note'. Sorry about that.