dusky - 2:55 am on Jul 1, 2010 (gmt 0)

Plain and simple, before caffeine, their index was not coping, and needed help from us to minimize on "thin / non essential internal pages" and to use their index space only for the best content any site could offer. Gbot was getting stuck in the loop of the so called duplicate and useless content (printer friendly pages, user profiles, logins, newsletter sub forms, reply to threads links etc, etc..). Now they have caffeine (much larger space to store), faster crawler and indexing system, and a "smarter" algo probably one of its best points is to accurately index the intended page and leave out other dups of it with ease. What else to do except come out with a public announcement on a blog disguised as a user asking a question (as usual from MC).

All of a sudden, now it's okey to let printer friendly forum / news pages for example be indexed and followed and it's fine if they get into the index!

They caused the knee jerk reaction in the first place with all sorts of scare tactics when actually they only wanted us to help them STOP their crawler collecting duplicate and non-essential content, not because of pagerank or any other ranking factor!

Surprise! surprise! the announcement came after an almost complete halt of Gbot activity for two days. Add to that, their hint on site: and inurl: commands and the zillions of phantom backlinks and internal links which are years old, long deleted, dups or blocked by robots.txt file on WMT.
They want us to open every link to every page on our sites for indexing, they want to decide which to index and which to ignore, they even rolled out years old indexed historical data and sent gbot crawling every long dead link just to prove a point, the problem is, that point is top secret!