Google and robots.txt

90 day wait issue

[Blade]

Am I reading this incorrect or does Google stop indexing a site for a 90 day period when using a full exclusion in the robots.txt file.

[google.com...]

For me, I would not want indexed until I have enough sites linking to mine, but couldn't afford the 90 day wait!

Anyone confirm this?

[agerhart]

That's the way it sounds to me.

If you don't want your site indexed by Google, you can either disallow Googlebot in the robots.txt, or just don't launch your site until it is done.

[Brien]

An alternative would be to place the following on each page:

<meta name="robots" content="noindex,nofollow,noarchive">

This would exclude just this page, but Google would keep coming back to reindex new pages.

To test google's frequency of index you could create a page that does NOT have that tag in it, and place NO keywords or even words at all, so that it would not show up in any real searches. Make it an asp page to store the date in a non-standard format and you will be able to poor-man track when google last hit and atrchived your page by using a google query like "site:blah.com -the", and click on the archived google version to see when they were there last.

That would be to plan when to remove the individual page deference tags in anticipation of going live...