| Welcome to WebmasterWorld Guest from 184.108.40.206 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|Matching patterns in robots.txt|
I have two dynamic URL pages;
I want to allow robots to crawl the first page but i don't want robots to crawl the page with "&start"...How can i do this.
If I use
"Disallow: /index?id" will block both the URL patterns. So How can i be specific..
In my robots.txt:
I have added,
Is this correct....
Please help me..
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved