| Welcome to WebmasterWorld Guest from 18.104.22.168 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Blocking Google Only|
From certain sub domains
| 1:54 pm on Apr 10, 2007 (gmt 0)|
Being a victim of Google's new algo, I need to stop them (Google) from crawling certain sub directories of my site. How do I do this?
| 1:14 pm on Apr 11, 2007 (gmt 0)|
Add this to your robots.txt
To verify your robots.txt is correct you can use Google Webmaster Central.
| 7:06 am on Apr 13, 2007 (gmt 0)|
Thanks, but Google's own pages say to leave sub-directories /open without the traling backslash, no /closed/ as with everyone else.
| 7:18 pm on Apr 13, 2007 (gmt 0)|
to further complicate things you can look at their own robots.txt file (http://www.google.com/robots.txt) which uses both styles :)
| 7:25 pm on Apr 13, 2007 (gmt 0)|
I believe there is a difference between the two styles (please correct me if I'm wrong!). As I understand it:
Matches the directory only.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved