Forum Moderators: open
I have come up with a theory that Google is spidering websites and web pages that were visited with browsers that have the Google Toolbar installed.
At the begining of June I notices an entry in my logfiles that Google had spidered a page that has no links to it (unless a user submits a form), see my thread at [webmasterworld.com...]
Today I noticed that Google has indexed a site that I am working on. This is a brand new site and domain which has no external links yet and was not submitted to any search engine or directory (the site is still under production, hence no effort was put into the link exchanges and SE submission yet).
The only thing in common between the two sites that would "notify" Google about them, that I can think of, is the fact that I have the toolbar installed and was at those pages/sites.
Any thoughts or ideas?
Alon
Traffic logs are one way spiders can find unlinked, unsubmitted sites. If you browse from the "hidden" page to another site, that URL may end up in the other site's logs or traffic reports.
If an "advanced mode" Toolbar user returns a URL to them that is not already indexed why shouldn't they add it?
If you don't want a URL to be discovered through any means then you should take steps to prevent access to the public until you're ready.
Literally 4 or 5 months up and running - me viewing with toolbar installed - and no Google index. As soon as I linked to it from our main site, it got gobbled up.
Im pretty sure the Google Toolbar doesnt get your site listed.
The Alexa toolbar however - ranks your site - lists your site - Google indexes Alexa....possiblity, no?
Scott