Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How do I not allow google

9:22 pm on Oct 3, 2007 (gmt 0)

Junior Member

10+ Year Member

joined:May 14, 2007
votes: 0

Hi folks, a new section is coming for my web site. It is not finished yet and I would like some of my visitors to access this page so that I can figure out what else I have to do on the web site according to this info.

The only thing is I want this page to be accessible by my visitors but I do not want Google or other robots to index these pages. How do I allow this? I read that no index tag works but google will ignore it. Is that true? I would appreciate if somebody can advice me about this.

Thanks in advance

10:31 pm on Oct 3, 2007 (gmt 0)

Senior Member

WebmasterWorld Senior Member jimbeetle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Oct 26, 2002
votes: 9

What Gbot sometimes appears to "ignore" is a disallow directive in robots.txt. If the 'bot finds enough links to a disallowed page it thinks the page is somewhat important and will include it in the index as a "URL only" listing, basically naked with not title or description.

To be sure a page isn't included the meta noindex is the best bet. Just be sure not to also disallow the section in robots.txt as the 'bot has to be able to read the pages in order to obey the noindex.

I'd probably also slap a nofollow on it:


I used the generic ROBOTS instead of the specific GOOGLEBOT because, well, you know, there are other search engines out there ;-).