Msg#: 23011 posted 2:52 am on Mar 31, 2004 (gmt 0)
Is that true that Google is now spidering pages that have no inbound links and are not even linked in the navigation? I heard so rumors but I am just curious if that is true and if it is how do they do it?
Msg#: 23011 posted 9:50 am on Mar 31, 2004 (gmt 0)
Google can find pages in one (or more) of these ways:
- link on another page - link found in referer log of a site the page links to (you probably don't know the link is there!) - someone who installed the googlebar viewed the page. Googlebar then reported the existance and viewing of the page back to Google.
Msg#: 23011 posted 10:12 am on Mar 31, 2004 (gmt 0)
A site like Googledorks (search for the link in google) is dedicated to reviling searches in Google that will show confidential information. This information is practically never linked to from other pages
Msg#: 23011 posted 12:59 pm on Mar 31, 2004 (gmt 0)
I have a new site, with two main purposes - one is a forum for a group to which I belong; there are no links to this forum. The users know the URL as www.mydomain.com/forumname, whereas I have subsequently created a subdomain to access the forum as forumname.mydomain.com. The other part of my site is a public website. I've been waiting hopefully for Google to come crawl my main site, so far no luck. But wouldn't you know it, the first referral I got from Google came from a search for the private forum! Google referred it as the subdomain url. No idea how they found it. Rob
Msg#: 23011 posted 5:59 pm on Mar 31, 2004 (gmt 0)
I have seen Google spidering the sites that i used in a form's post address..e.g. On my index page, I have an input box and the form is submitted to www2.mysite.com/form.php OR submitted to www3.mysite.com/form.php. Everytime the page loads, I show randomly either www2 or www3. Google has crawled the index page of both www2 & www3 sites. There is NO link to these pages from anywhere else in the web world.