Welcome to WebmasterWorld Guest from 54.162.226.212

Forum Moderators: goodroi

Message Too Old, No Replies

How do I not allow google

     
9:22 pm on Oct 3, 2007 (gmt 0)

5+ Year Member



Hi folks, a new section is coming for my web site. It is not finished yet and I would like some of my visitors to access this page so that I can figure out what else I have to do on the web site according to this info.

The only thing is I want this page to be accessible by my visitors but I do not want Google or other robots to index these pages. How do I allow this? I read that no index tag works but google will ignore it. Is that true? I would appreciate if somebody can advice me about this.

Thanks in advance

10:31 pm on Oct 3, 2007 (gmt 0)

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member



What Gbot sometimes appears to "ignore" is a disallow directive in robots.txt. If the 'bot finds enough links to a disallowed page it thinks the page is somewhat important and will include it in the index as a "URL only" listing, basically naked with not title or description.

To be sure a page isn't included the meta noindex is the best bet. Just be sure not to also disallow the section in robots.txt as the 'bot has to be able to read the pages in order to obey the noindex.

I'd probably also slap a nofollow on it:

<META name="ROBOTS" content="NOINDEX,NOFOLLOW">

I used the generic ROBOTS instead of the specific GOOGLEBOT because, well, you know, there are other search engines out there ;-).

 

Featured Threads

Hot Threads This Week

Hot Threads This Month