Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Google Index robots.txt


webseosolution - 10:26 am on Aug 24, 2012 (gmt 0)


Thank you for your valued response.

Those files were accessible earlier, so those were crawled earlier.

Currently I have made them paid, that's why.

But some of the urls are having https:// even though they are being crawled by google bot.



WSS


Thread source:: http://www.webmasterworld.com/robots_txt/4487907.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com