| Welcome to WebmasterWorld Guest from 126.96.36.199 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
nervous beginner seeks reassurance
| 5:34 am on Aug 11, 2002 (gmt 0)|
I have read some messages here about the robots.txt file, and understand it is vital to syntax it correctly.
Being a newcomer to the use of the robots.txt file, can someone please reassure me that the following is correct:
The aim is to stop Googlebot trying to look at the page Changelanggen.asp, and the text file robots.txt is placed in the root directory.
Appreciate any comments.
| 6:04 am on Aug 11, 2002 (gmt 0)|
Hello jgar, you said...
Yes, that is correct. All other files within that directory will be indexed. Root directory is the only place you can put your robots.txt file.
| 7:06 am on Aug 11, 2002 (gmt 0)|
I'm also new to using robots.txt and I found the validator provided by webmasterworld greatly helpful. Take a look here [searchengineworld.com...] :)
| 7:30 am on Aug 11, 2002 (gmt 0)|
Thanks for the tips - the validator is a great resource
| 4:51 pm on Aug 12, 2002 (gmt 0)|
Just one extra comment.
robots.txt is case sensitive, so "Disallow: /Application" disallows anything beginning with "/Application" but not "/application".
Has anyone checked that Google does this right?
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved