Forum Moderators: open

Message Too Old, No Replies

Googlebot stopped visiting

should I be concerned?

         

jimone

9:31 am on Jul 27, 2002 (gmt 0)



Googlebot showed up every day in my web site's stats until July 16.
I have not seen Googlebot since that day.

What could have happened here?
Are there more people who miss Googlebot?

misosoph

10:00 am on Jul 27, 2002 (gmt 0)

10+ Year Member



The same has happened with my site. I am not sure about the date, but it seems over a week since the googlebot has come. It used to come everyday for my pages that are listed in yahoo.

I have no idea why. I hope it wasn't something I said! But if the same has happened with your site, then maybe it is nothing personal. (It is worrying, though.)

ciml

1:22 pm on Jul 27, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Welcome to WebmasterWorld, jimone.
(Make sure to check out paynt's welcome post [webmasterworld.com])

I wouldn't worry yet. I have some sites getting spidered hard, but others not touched yet in the new crawl.

If you're in Yahoo! or the Open Directory then at least one page should be fetched more than once per week.

misosoph

6:50 pm on Jul 27, 2002 (gmt 0)

10+ Year Member



Hi, I took jimone to mean that googlebot has not dropped by for almost two weeks. When you're used -- (in my case) for nearly a year -- to having it come every day (daily), then it is disturbing that it has disappeared.

I also have not seen a trace of the googlebot -- not one single hit -- for over a week.

The only reason I am not worried is that I haven't changed anything fundamental about my site for many months.

If jimone has made fundamental changes that may make googlebot's absence very worrisome. (I hope I did not appear to make light of that.)

<added>
Of course, if only jimone and I are seeing googlebot's absence, then I will start to worry. Isn't anyone else seeing this in their logs? I added this before seeing the next post.
</added>

[edited by: misosoph at 8:58 pm (utc) on July 27, 2002]

rfgdxm1

8:53 pm on Jul 27, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Be afraid jimone. Be VERY afraid. This inexplicably happened with my main site on the 14th, and in the last index Google dropped it (except, oddly some "Apache HTTP Server Version 1.3" manual files that are indexed.) The Google toolbar now is gray for my site. And, for anyone interested this site is in *both* the DMOZ and Yahoo directory. I've e-mailed Google about this. I am curious if I'll ever get more back than that "Keep on Googlin'" auto-ack. I have no idea what happened, but all I can say is this site wasn't using anything like keyword stuffing, participating in link farms, etc. that might get it the Google Death Penalty.

misosoph

9:06 pm on Jul 27, 2002 (gmt 0)

10+ Year Member



rfgdxm1,

I just looked for my site in Google, and everything looks normal there (in fact, one of my pages moved up from 8th on the page to 4th). Maybe the bad is yet to come.

I have no idea about my site's PR per se; I've never bothered to learn about it. It's not a commercial site.

jimone

2:13 pm on Jul 28, 2002 (gmt 0)



thanks for your reactions.
i'll wait and see.

cYbErDaRk

3:09 pm on Jul 28, 2002 (gmt 0)

10+ Year Member



Cry with me, cry.

On google's last update I've lost 26000 indexed pages, just showing now 50 (yes, 50).

:( :(

rfgdxm1

5:53 pm on Jul 28, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I feel your pain, cYbErDaRk. In my case Google managed to lose the whole site, even though it is in the DMOZ (which is also the Google directory), and the Yahoo directory. Yahoo is now using Google for their searches. Google seems sufficiently inept on occasion they can't even managed to include sites in their own, and Yahoo's directory.

I guess you can hope the 25,950 other pages will be back next month. :(

cYbErDaRk

6:08 pm on Jul 28, 2002 (gmt 0)

10+ Year Member



Thanks rfgdxm1

It's quite annoying, it's the work of two years, trying to do things well, to have it directly thrown to the trash in just one day...

I've write to help@google, let's pray..

Regards (and sorry for my English)

mahlon2

7:24 pm on Jul 28, 2002 (gmt 0)

10+ Year Member



It's out looking for invisible text!

rfgdxm1

8:31 pm on Jul 28, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I know cYbErDaRk. :( I've written Google too, but I really suspect that they after they send the "Keep on Googlin'" auto-ack, they probably just send all e-mail to /dev/null/. And, Googlebot hasn't been around for 2 weeks, and it used to come around every 2 days or so for a nip. I have a feeling somehow, someway Google has thrown my site in the trash, and possibly for good. Things are looking so bleak I'm seriously considering just abandoning the current domain, moving on and hoping for the best. The really weird thing is, while I could understand some computer bug causing a site to not be in the index a month, WHY no more visits from Googlebot with a site in both the Yahoo and DMOZ directory? Remember the Google directory is just the DMOZ. My site *is* in the Google directory, yet not in Google!?

HackingLawyer

8:34 pm on Jul 28, 2002 (gmt 0)

10+ Year Member



How can you tell if Googlebot has visited. My webtrends report shows googlebot in the browser section instead of the spider section.

Also, how can you tell how many pages google has indexed.

Thanks

misosoph

8:34 pm on Jul 28, 2002 (gmt 0)

10+ Year Member



<edit>Posted before seeing the last two messages</edit>

Google looks to be facing the same fate as Microsoft -- to be kowtowed to, feared and hated. Microsoft makes some good software and Google does some good searches -- but only a fool loves a monopoly or a dictatorship (because what it exalts today it can just as easily crush tomorrow, and there is no recourse).

cYbErDaRk, your English is far better than most people's here (including mine). :)

rfgdxm1

8:51 pm on Jul 28, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You got it right, misosoph. What is the best strategy to kowtow to Google at the moment? :(

yankee

10:24 pm on Jul 28, 2002 (gmt 0)

10+ Year Member



Same thing here. I used to see googlebot almost everyday, but I've only been visited on 2 of the last 13 days. My sites last month got spidered five days after the dance started, so I expect the crawling to start any day now.

jady

12:29 am on Jul 29, 2002 (gmt 0)

10+ Year Member



Jimone - I am in the exact same boat. Usually just after the update or even during the update, the bot begins deep crawling. Havent seen her yet...

Dpeper

12:33 am on Jul 29, 2002 (gmt 0)

10+ Year Member



Did you guys stop leaving milk and cookies out?

lazyz

3:32 am on Jul 29, 2002 (gmt 0)

10+ Year Member



Jady - same deal here... Usually I see Googlebot during or immediately after the update. No googlebot. I have several sites on several different servers. Maybe Googlebot is sucking up more of the web and taking longer to get to me. Even when I had some PR0 sites, Googlebot came to visit at least once to tease me.

misosoph

5:04 am on Jul 29, 2002 (gmt 0)

10+ Year Member



HackingLawyer wrote (msg #:13 above):

How can you tell if Googlebot has visited. My webtrends report shows googlebot in the browser section instead of the spider section.

Also, how can you tell how many pages google has indexed.

I don't know whether anyone else is going to respond to your questions is this thread. Unfortunately I am unfamiliar with webtrends reports. What I can say is that googlebot is Google's robot/spider, and so if it is showing up in your logs, then your site is being spidered. I've never seen or heard of googlebot being used to do anything else (But I'm not a Google expert).

This is what our host says:

Don't ask questions in the middle of another thread. If you have a question (other than a clarifing question that is on topic to the current thread), then please start a new thread.

If you don't, usually your questions won't get answered. At least that's my observation. I hope that helps a little.

rfgdxm1 (msg #:15), if I knew that, I'd be as rich as the "Man who Broke the Bank at Monte Carlo". -- Of course, "... who Broke the Google Algorithm" sounds geeky; who would write a song about it?

yankee, jady, lazyz -- Your notes are reassuring. But only if your sites are still listed in the Google index. Are they?

yankee

5:48 am on Jul 29, 2002 (gmt 0)

10+ Year Member



Yes, the sites are in the new index and haven't been crawled yet. I think the major crawl will start in the next day or two, August 1st at the latest.

lazyz

4:20 pm on Jul 29, 2002 (gmt 0)

10+ Year Member



misosoph - yes, my sites are stronger than ever this index. New pages added and updated cache.

misosoph

4:43 pm on Jul 29, 2002 (gmt 0)

10+ Year Member



Thank you, yankee and lazyz. I never understood why the googlebot used to come every day, anyway. What was the point?

If Google was only looking for dead links, then surely it could not have been visiting every site on the Internet every day! And since the visits did not result in the Google cache being updated -- Again, what was the point?

lazyz

2:43 am on Jul 30, 2002 (gmt 0)

10+ Year Member



Googlebot just had it's first taste of one of my domains...
Not deep, just the index and robots.txt file.

jimone

1:29 pm on Jul 31, 2002 (gmt 0)



as i mentioned on July 27, Googlebot stopped visiting my site on July 16

now i can tell you that Googlebot came back on July 30, pm 23h.

so it seems that Googlebot toke a 14 days holiday.

i am glad it is back and hopefully stays coming back.

------------------

last month i noticed another curious thing about Google.
the homepage of another web site of mine, had a PR7.

although the page was still listed in Google,
there was NO pagerank anymore after a google-update.
(may be due to the fact that in the beginning the pagecontent was somewhat similar to a page on another website of mine;
i changed the content afterthat)

then, after the latest update (July 28?) it has PR2 !

------------------

regards to all,
jimone