| Welcome to WebmasterWorld Guest from 188.8.131.52 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Quick question about robots.txt|
Excluding dynamic pages
is this going to be enough to exclude robots from a dynamic page?
it will stop any robots that obey robots.txt. If you want to do any more you will need to look at htaccess.
Nope, just google mainly :) Maybe I need to clarify,
Going to stop a robot that obeys robots.txt indexing something like this: dynamicpage.asp?something=something
I'm assuming that it will, but just want to make sure. Thanks.
That may not stop anything at all... But hopefully, the missing ":" after "User-agent" was a typo. :)
Will stop any robot that obeys robots.txt from indexing something like this: dynamicpage.asp?anything
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved