I have an interesting situation in that Google is listing Urls in search which it can't actually get to. The pages are intranet pages in an organisation and the IP access to those pages is strictly limited to IPs within the organisation.
The pages have robots meta tag and http header tag - but of course these are irrelevant because the spider doesn't get anything other than the domain is unavailable because the request is blocked
I'm assuming that this is due to people inside the firewall using Chrome to access the pages and then chrome sending that data back to HQ and the index then having a record of the URL.
We are planning on ensuring that all chrome instances in the organisation are set to not send data back. Are there any other measures we can take to ensure that these URLs are not listed in Google SERPs