---- "Extremely high number of urls" report includes robots.txt urls
indyank - 1:20 pm on Sep 16, 2011 (gmt 0)
What's puzzling is that the list of examples includes lots of URLs that are either excluded via our Robots.txt file or use parameters that should be ignored based on our parameter handling settings.
They should be obeying your robots.txt unless you have used it wrongly. But, I have never seen their bots respecting those "parameter handling settings" in google webmaster tools.I don't even know why they provide that feature when their bots attempt to crawl most of those URLs.
The only solution to block them is robots.txt. But they find workarounds for that these days. For example, make sure that you don't have a +1 button on those pages. otherwise, they might not obey robots.txt