Forum Moderators: open

Message Too Old, No Replies

Google Spidering Sites/Pages visited using the Toolbar

pages were not submitted to ANY SE and no backlinks

         

paladin

2:28 pm on Jun 20, 2003 (gmt 0)

10+ Year Member



Hi All,
My apologies if this has already been discussed in the past.

I have come up with a theory that Google is spidering websites and web pages that were visited with browsers that have the Google Toolbar installed.

At the begining of June I notices an entry in my logfiles that Google had spidered a page that has no links to it (unless a user submits a form), see my thread at [webmasterworld.com...]

Today I noticed that Google has indexed a site that I am working on. This is a brand new site and domain which has no external links yet and was not submitted to any search engine or directory (the site is still under production, hence no effort was put into the link exchanges and SE submission yet).

The only thing in common between the two sites that would "notify" Google about them, that I can think of, is the fact that I have the toolbar installed and was at those pages/sites.

Any thoughts or ideas?

Alon

rogerd

2:31 pm on Jun 20, 2003 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Paladin, GoogleGuy has said in the past that Google didn't use toolbar data to find new URLs, though he didn't state that this would always be the case.

Traffic logs are one way spiders can find unlinked, unsubmitted sites. If you browse from the "hidden" page to another site, that URL may end up in the other site's logs or traffic reports.

dmorison

2:35 pm on Jun 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Nothing to get too excited about, Google want to the provide the biggest and best possible search experience, and to that end they need to cover as much of the web as possible - and should therefore use any source of URLs available to them!

If an "advanced mode" Toolbar user returns a URL to them that is not already indexed why shouldn't they add it?

If you don't want a URL to be discovered through any means then you should take steps to prevent access to the public until you're ready.

Marketing Guy

2:41 pm on Jun 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I had a semi complete site up for months that I didnt link to from anywhere or submit to any directories, etc.

Literally 4 or 5 months up and running - me viewing with toolbar installed - and no Google index. As soon as I linked to it from our main site, it got gobbled up.

Im pretty sure the Google Toolbar doesnt get your site listed.

The Alexa toolbar however - ranks your site - lists your site - Google indexes Alexa....possiblity, no?

Scott