Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- faulty robots.txt?


tenaka - 9:27 pm on Jan 2, 2004 (gmt 0)


thx guys,

I just got an email from inktomis tech support telling me that they are retrieving the robots.txt from time to time to check the pages are still up. They also told me that they have thousands of indexed pages from my cgi-bin that I banned them from recently because that was a mistake on my side. well, actually they said they have a few pages from my site in their index and if I have more I should link to them.

I checked again and they only have 3 pages and 1000 of old ones from my cgi-bin...

I will just wait a little longer...


Thread source:: http://www.webmasterworld.com/robots_txt/222.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com