Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Robots.txt returning "test"


operafan - 10:37 am on Jan 28, 2004 (gmt 0)


If you're on a virtual host & have ftp access, you should know wheter there's a robots text in it.
Robots txt cant do much in a way to block your index page being spidered.
If you believe your host is blocking se crawlers than i believe they might have done it on the main server settings, conf if on apache.. dont know about what ink is talking about though.
If it's a robot.txt you surely can see what's in there.
I've just checked your site profile /robots.txt & i get a 404 means it's missing.
Then I suspect if what the 2 ppl/ink is saying when they see test when they request for your robots.txt is probaly your host provider have set a rewrite rule for anybody requesting the robots txt to display test.


Thread source:: http://www.webmasterworld.com/robots_txt/256.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com