Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- robots.txt Disallow - How much time till its obeyed?


notsleepy - 1:51 pm on Aug 6, 2003 (gmt 0)


I've created a new page my-page.htm that I do not want to be included in the Google index.

I've added the exclusion in my robots.txt:
User-agent: *
Disallow: /my-page.htm

However I haven't uploaded the file yet b/c I know it takes some time for all googlebots to get word that they should not follow links to that page.

Does anyone have an estimate on how long I should wait before uploading the page?


Thread source:: http://www.webmasterworld.com/robots_txt/32.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com