Page is a not externally linkable
MHes - 8:53 pm on Nov 27, 2012 (gmt 0)
Many thanks Deadsea. I have many generated pages (2.3 million indexed) that will change depending on the user agent, and about 80 that won't. These pages that won't change are rarely visited but important for the spider to navigate my site.
Someone suggested I generate the headers via the perl scripts that generate the pages on the fly, but I have no idea how to do this! The safer option seems to be to set the Vary http header on the server for all the pages. I know you are right about being able to set them for specific pages, but there are too many to list..... or am I missing something? (Way out of my depth on this one!)