Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Googlebot possibly ignoring robots.txt


Andem - 9:18 pm on Apr 26, 2012 (gmt 0)


>> First, I just want to mention a friendly reminder that robots.txt is a voluntary protocol.

Well, that may be true but I'm giving Google and anybody else specific instructions not to index and/or scrape my content.

I pasted the contents of my robots.txt file in the OP. Did I miss something?

Cheers


Thread source:: http://www.webmasterworld.com/robots_txt/4446154.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com