I recently came across a site that seems to be serving limited functionality to search engine user agents. I have always been extremely cautious when serving different versions of a site based the user agent for obvious reasons. In this example content and pages are identical so I don’t believe they are in any way fooling engines for potential gains.
In the example I have seen the functionality that is restricted is limited to the sites search facility and check out process.
My first question is why anyone would do this when pages from the aforementioned areas of a site can be excluded via robots protocol?
My second question is whether anyone believes a penalty can be attached to this, when in effect you are only restricting access areas of the site search engines wouldn’t need to index?
Thanks in advance
