sftriman - 10:33 pm on Mar 27, 2012 (gmt 0) [edited by: tedster at 12:57 am (utc) on Mar 28, 2012]
I posted about this a few months ago, but I want to start a new thread to consider possible new ideas.
I have a search page, say search.php, that returns dynamic results. It is a GET, not a POST. Apparently, that was one of my first mistakes, yet I do note that many sites use GET for their search functionality. Anyhow, I do have some links on my site to popular searches - maybe 100 or so over time. I did NOT use the canonical tag, and hence I had many drilldown and sort options indexed by Googlebot. My bad.
However, I did put rel="nofollow" on all those links, but that was ignored. So now, I have the robots meta tag to "noindex" for any search except a plain old keyword search. I will note that in another thread, I was cautioned to completely block search from Google indexing. But when I compare traffic for search.php?q=brandx to my /brand/brandx flat HTML page, the search page gets way more traffic.
So I'm inclined to not 301 redirect search to brand pages (or other) or to "noindex" all my search pages just yet. Still, what I've done in terms of my internal searches and what I consider valid searches will certainly help things out.
What really irks me, though, is all the random searches to my site. If I'm selling baseball items, why do I get 100's of searches a day for topiary items, bathroom products, etc. etc. etc.? I feel like Googlebot churns making all these useless hits on my site that aren't relevant. I can't track where they are coming from. Someone said, from a search toolbar. That doesn't entirely make sense - why would such searches lead to my site?
The bottom line is, I want to get rid of all these junk searches. What I've done is capture 14,000 of them (yes, 14,000!)
Now, at the top of my search.php code, I check the search, and if it matches one of these weird searches, I return an empty page with a 410 header. UNLESS, as I've hard-coded, there is a single word in a weird search which has relevance to my site, in which case I search that single word, show products, return a usual 200 page found header, but put in the robots noindex meta tag to stop the page from being crawled.
All said, I think these searches are a waste of my time, Google's time, and have no value to my site. Plus, I think they are bogging down my genuine listings in Google.
Can someone suggest how to "get rid of" all these bogus searches that are coming from Googlebot?
[edit reason] no specific keywords, please [/edit]
[edited by: tedster at 12:57 am (utc) on Mar 28, 2012]