Page is a not externally linkable
-- Content, Writing and Copyright
---- Rat Trap for Content Scraper
kwasher - 2:04 am on Aug 15, 2007 (gmt 0)Thread source:: http://www.webmasterworld.com/content_copywriting/3418558.htm
|if robots.txt obeying bot comes to your site and see that the page is disallowed, it will never try to fetch it |
Are you sure about that?
Brought to you by WebmasterWorld: http://www.webmasterworld.com