Forum Moderators: open

Message Too Old, No Replies

Format of robots.txt file to exclude Yahoo slurp

robots txt file to exlcude yahoo slurp

         

fred9989

10:21 am on Apr 30, 2004 (gmt 0)

10+ Year Member



Can a more knowledgeable expert (more knowledgeable than me, that is) confirm if a robots.txt file designed to exclude Yahoo's robot should read:
User-agent: Slurp
Disallow: /
OR
User-agent: Yahooslurp
Disallow: /
OR even should be something else altogether?
Thanks
Rod

Yidaki

10:29 am on Apr 30, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member




User-agent: Slurp
Disallow: /

Yep, this should work.

fred9989

4:39 pm on Apr 30, 2004 (gmt 0)

10+ Year Member



Thanks, Yidaki