I've always wondered how do you know if a search engine (Google)has spidered your site. Does typing your site in give you any idea?
9:08 am on Sep 16, 2005 (gmt 0)
I think most people know by looking at their RAW server log files. It's pretty easy to figure out. If you don't have access to these files, usually you can see search engine user agents in stats packages most hosting providers offer for free.
6:02 pm on Sep 16, 2005 (gmt 0)
Speaking of spidering... I see that G spiders one of my sites more frequently than the other. The one that is less spidered is older and less active. When I look at the page views, neither of them is really popular altho both have a PR of 4/10. I just added AdSense and have been really busy building up content pages. I add at least a page a day. I have a third site that is just not even listed by Google although I have had it for a couple of months. I finally requested Google to spider it a couple of weeks ago. Why would the older site be spidered less than the newer site -- G has not had time to know that I am adding content so frequently and both have about 20 pages. Well the second one has about 40 pages now.
As for the newest one...it is a .biz site rather than .com....hmm, just realized the older but less spidered page is .ca. Could this be a factor?