Page is a not externally linkable
- Google
-- Google SEO News and Discussion
---- How has Google found a page with no links to it?


Robert_Charlton - 8:47 pm on Jul 21, 2012 (gmt 0)


My guess would be not the toolbar, but publicly available server logs. In the past, Google has denied that the Toolbar was involved, and currently it appears that (by omission) they would perhaps be denying the same thing about Chrome. I'm including dates in the references below to indicate the history. We had a discussion about a similar problem here. Note the Google FAQ quote I posted about the "secret" web server...

Why is Google indexing my entire web server?
July, 2007
http://www.webmasterworld.com/google/3396393.htm [webmasterworld.com]

That quote has gone 404 several times, but was quoted specifically by Matt Cutts on his blog (note the title of the article)...

Generic Toolbar Indexing Debunk Post
by Matt Cutts on July 19, 2008
http://www.mattcutts.com/blog/toolbar-indexing-debunk-post/ [mattcutts.com]

The Google FAQ link is also broken in Matt's post. I'll quote it here....

Why is Googlebot downloading information from our "secret" web server?

It’s almost impossible to keep a web server secret by not publishing any links to it. As soon as someone follows a link from your "secret" server to another web server, your "secret" URL may appear in the referrer tag and can be stored and published by the other web server in its referrer log. So, if there’s a link to your "secret" web server or page on the web anywhere, it’s likely that Googlebot and other web crawlers will find it.

Part of this quote is now incorporated into current Google WMT documentation...

Googlebot - Webmaster Tools Help
updated 06/07/2012
[support.google.com...]

Chrome isn't mentioned. Can't say whether they would use it for Googlebot discovery or not.

I have very mixed thoughts about Google's aggressive indexing, btw. As a web professional who knows what I want indexed and what I don't, Google's aggressiveness in indexing has been a PITA. As a searcher looking for important information where webmasters have been too inept to make it visible, I can understand what Google's doing, and occasionally I've been glad they've done it.

Bottom line, if you don't want something indexed, use noindex and/or password protection.


Thread source:: http://www.webmasterworld.com/google/4477784.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com