Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Fake user agent strings


wilderness - 6:39 pm on Jun 11, 2012 (gmt 0)


Disagree. It's what the HTML standard recommends.


keyplr,
We'll need to agree to disagree on this issue.

I'm sure HTMl standards also advise the allowing of cache on web pages, which I also do not.

Just today I had some Oceanic visitors (denied range) looking for a particular image.
Since they couldn't gain access, two hours later here comes the same request from a local universities IP range, of which the later is also denied.

I've approximately 4,000 images on one website. Some pages have fifty or more thumbnails.
To allow mass access to all these images would be absurd.
Redirecting same visitors to a presentation of a fee-based possibility would also be absurd, at least for widget folks.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4462691.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com