- - Google
- -- Google SEO News and Discussion
- ---- Several Parameters Indexed in URL
phranque - 8:24 am on Apr 17, 2013 (gmt 0)
i would suggest adding a meta robots noindex element to your search pages and allow them to be crawled.
starting about 6 months ago google changed the way search results snippets are shown for urls excluded from crawling.
|A description for this result is not available because of this site's robots.txt – learn more |
the only good reasons i can think of for excluding crawlers with robots.txt is to preserve crawl budget and to operate honeypots.
Thread source: http://www.webmasterworld.com/google/4565203.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com