| Welcome to WebmasterWorld Guest from 22.214.171.124 |
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
|Pubcon Platinum Sponsor 2014|
|How to disallow by specific keyword?|
| 9:45 am on Mar 27, 2014 (gmt 0)|
I want to block all pages on my site that include the word /tabid/
So if I add Disallow: /tabid/ to my robots.txt will this disable all URLs levels for example:
Or what is the correct way to Disallow by keyword - in this case "tabid", located in different places in my URLs?
| 11:17 am on Mar 27, 2014 (gmt 0)|
the Disallow directive matches the specified pattern from left-to-right.
for those bots which support wildcard extensions you can use:
| 12:20 pm on Mar 27, 2014 (gmt 0)|
As pranque said, correct, just bear in mind that not all spiders are that smart (ie, wildcards). If concern is Google, Bing or Yahoo, then you are good.
| 12:29 pm on Mar 27, 2014 (gmt 0)|
| 7:04 am on Jun 11, 2014 (gmt 0)|
There is no another way to block the URl'S. You can just remove the tabid then the URL are dissapled.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved