Welcome to WebmasterWorld Guest from 54.144.77.26

Forum Moderators: goodroi

Message Too Old, No Replies

How to disallow by specific keyword?

     

virtualreality

9:45 am on Mar 27, 2014 (gmt 0)

5+ Year Member



I want to block all pages on my site that include the word /tabid/

So if I add Disallow: /tabid/ to my robots.txt will this disable all URLs levels for example:

www.site.com/folder/tabit/folded.aspx
www.site.com/folder/folder/tabid/file.aspx
www.site.com/tabid/folder/folder/file.aspx
etc..

Or what is the correct way to Disallow by keyword - in this case "tabid", located in different places in my URLs?

Thanks!

phranque

11:17 am on Mar 27, 2014 (gmt 0)

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



the Disallow directive matches the specified pattern from left-to-right.

for those bots which support wildcard extensions you can use:
Disallow: *tabid

tangor

12:20 pm on Mar 27, 2014 (gmt 0)

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



As pranque said, correct, just bear in mind that not all spiders are that smart (ie, wildcards). If concern is Google, Bing or Yahoo, then you are good.

virtualreality

12:29 pm on Mar 27, 2014 (gmt 0)

5+ Year Member



Many thanks!

Hemanth Malli

7:04 am on Jun 11, 2014 (gmt 0)



Hi,
There is no another way to block the URl'S. You can just remove the tabid then the URL are dissapled.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month