| Welcome to WebmasterWorld Guest from 18.104.22.168 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Order of lines in robots.txt file|
Can you control a bot with the order of the lines?
| 9:59 pm on Sep 20, 2006 (gmt 0)|
This is a sample from my robots.txt file.
What I am trying to achieve is this;
I want Google to follow links with "action=this_link" in the URL. But I want to block all other links with the term "action=" in the URL.
Am I right in thinking that, if I put the 'Allow' line first, that I will achive my aim?
| 3:05 pm on Sep 22, 2006 (gmt 0)|
Using this robots.txt:
Will give you this result with Google:
| 3:37 am on Sep 23, 2006 (gmt 0)|
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved