Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Do Search Engine Robots Crawl Sites with no Domain Name?


montman - 10:52 pm on Dec 3, 2009 (gmt 0)


Over the course of about a week, and up until last Saturday evening, I was testing web pages on my ISP free hosting server. Sometimes I left the pages up for a few hours at a time before taking them down again. I was using this just for tests, and the pages were on my ISP without a domain name. I have since moved the pages to a hosting company with a domain name

Unfortunately, I was not aware of the ability to keep most robots from crawling the pages with a robot.txt file.

I am concerned that search engines might have crawled the pages. However, I'm hoping that because the pages did not have a domain name, and maybe even because the were on an ISP server, that it is unlikely they were crawled, and if crawled, unlikely they will be indexed. Are these good reasons why they probably wouldn't be crawled or indexed?


Thread source:: http://www.webmasterworld.com/robots_txt/4036537.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com