|Googlebot doesn't spider 1 particular site of mine...|
| 12:10 pm on Sep 22, 2004 (gmt 0)|
I run multiple websites, and all of them are listed in my "companies" portfolio. This portfolio get's spidered almost daily, so each new link I add will be visited by googlebot almost instantly. Then, after a few days, Googlebot will actually start indexing the subpages on this new domain as well.
However, I have one domain where this does not work for some reason. I have added it to my portfolio, waited for a while, but googlebot didn't even visit the new domain, eventhough it did visit my portfolio. Then I added a few links from other sites to this domain, but still a no-go.
Finally, I manually submitted the domain to google. And yes! Googlebot visited! However, it only hit 4 pages (where there are a lot more), and doesn't seem to be coming back... It has been quite a while now, so I am getting frustrated.
The only difference between this site/domain and the others I have launched, is the fact that this domain used to belong to another owner. It has been offline for quite a while as well. Then I took over and built a new site, which is not indexed like I wrote.
Any ideas on how this is caused? Any ideas on what I can do?
| 11:09 pm on Sep 22, 2004 (gmt 0)|
>The only difference between this site/domain and the others I have launched, is the fact that this domain used to belong to another owner. It has been offline for quite a while as well. Then I took over and built a new site, which is not indexed like I wrote.
Have you checked archive.org to see if the old site was somehow dodgy? It may still have a penalty.
| 11:39 pm on Sep 22, 2004 (gmt 0)|
designaweb, we have a similar experience. The most recent site we launched more than 10 weeks ago still hasn't been spidered by gooblebot. All the other search engines have already spidered it and revisit it daily. It's a subdomain of our main site, and other subdomains we launched before were spidered by Google within 2 to 4 weeks.
| 1:43 am on Sep 23, 2004 (gmt 0)|
I have a couple of sites on a particular host (not sure if I can name names). One site was doing well and is in the indexes. However, the other site went from about 100+ pages to 30 pages and traffic went to nothing.
I looked at a couple of things for the site that isn't doing anything. It is about 6 months old and as far as I know, I'm not doing anything ungooglelike.
I took a closer look at the august and sept logs and it appears that their is only one googlebot entry. (when I look at the logs from a site hosted elsewhere, I see hundreds of googlebot hits)
If I do a site:www.mysite.com -aasdasdas I see that only some of the listings are cached. The cache date is Aug 24. If I look at the log file, I would expect to have seen the googlebot in the august logs but it isn't there. If I look at July, I don't see googlebot there either (i do see the msnbot and a few others.)
I ran some robots.txt validators and robots.txt seems ok.
I tried the poodle predictor and that looks OK also.
I have a robots all,follow, index tag in all of the pages.
I did some google searches with my hostname to see if anyone else was complaining but didn't find anything.
Anyway, I'm a bit of a google novice. Here are a couple of questions
1. I'm assuming that not seeing the googlebot is really bad. Is this correct?
2. I tried a couple of sites where you could fake out the agent to see how the site reacts to the googlebot and I don't see any problems. Any suggests for sites to check my site out?
3. I'm not sure if it's within the TOS of this board to post a question about a specific host. I'd appreciate any thoughts on this.
I'd appreciate any ideas
| 1:53 am on Sep 23, 2004 (gmt 0)|
My sites are usually visited by Google daily, however the last 4-5 weeks nothing. About an hour ago all 10 of my sites are in the process of a deep crawl. So it appears whatever slow down there was has been put into high gear.
| 11:44 am on Sep 23, 2004 (gmt 0)|
Right, thanks for the reply's so far. I guess I will simply sit back and wait to see what happens over the next few weeks...
And yes, I did check archive.org - it seems to have been a mirror site for a site which currently is still online - could this cause anything to trigger google(bot)?
| 4:53 pm on Sep 23, 2004 (gmt 0)|
> it seems to have been a mirror site for a site which currently is still online
This could indeed explain it. If it was dupe content it got neglected in favor for the higher PR site. If it was a 301, does Googlebot stop visiting them at all? Or just look every while?
How long have you been waiting?
| 6:54 pm on Sep 23, 2004 (gmt 0)|
> How long have you been waiting?
Over a month now...