Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- Can I refuse Googlebot access via htaccess/ web.config?


scooterdude - 2:34 pm on Jan 3, 2013 (gmt 0)



Return the 4xx status code at the originally requested URL to signify that particular URL does not exist or no longer exists.


I now intend to return 4xx at the url as recommended by most,

However my conundrum is that the urls do exist

I want human users to use them,

But block googlebot/bingbot et all from visiting/indexing/requesting thes urls which are barred via robots.txt already.

Might G with its penchant for penalties one decide that this was cloaking?


Thread source:: http://www.webmasterworld.com/google/4532474.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com