| Welcome to WebmasterWorld Guest from 22.214.171.124 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Quick question about robots.txt|
Excluding dynamic pages
| 10:23 pm on Jan 14, 2003 (gmt 0)|
is this going to be enough to exclude robots from a dynamic page?
| 2:18 am on Jan 15, 2003 (gmt 0)|
it will stop any robots that obey robots.txt. If you want to do any more you will need to look at htaccess.
| 3:51 am on Jan 15, 2003 (gmt 0)|
Nope, just google mainly :) Maybe I need to clarify,
Going to stop a robot that obeys robots.txt indexing something like this: dynamicpage.asp?something=something
I'm assuming that it will, but just want to make sure. Thanks.
| 4:09 am on Jan 15, 2003 (gmt 0)|
That may not stop anything at all... But hopefully, the missing ":" after "User-agent" was a typo. :)
Will stop any robot that obeys robots.txt from indexing something like this: dynamicpage.asp?anything
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved