Welcome to WebmasterWorld Guest from 54.198.132.40

Forum Moderators: open

Message Too Old, No Replies

Googlebot still crunching?

yum yum yum

     
7:27 pm on Mar 1, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 5, 2002
posts:529
votes: 0


Hrmm ... Just checking my stats, and googlebot has visited my site 140 times in just 12 hours of this month ...

Is even the march 3rd guess too optimistic now?

12:39 am on Mar 2, 2003 (gmt 0)

Junior Member

joined:Feb 27, 2003
posts:88
votes: 0


March 3rd STRONG believer here.
3:39 am on Mar 2, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 7, 2003
posts:3
votes: 0


Where do you check your stats to see that Googlebot has visited your site?
3:56 am on Mar 2, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 28, 2003
posts:1978
votes: 0


Take a look at your log files for your site and you should be able to tell. Google disguises itself as 'googlebot'.
4:06 am on Mar 2, 2003 (gmt 0)

New User

10+ Year Member

joined:Jan 7, 2003
posts:3
votes: 0


Thanks bhartzer - where do I find the log files to my site? Is that the log from a Counter program, or is it something else?
4:25 am on Mar 2, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
posts:25430
votes: 0


Dieter,

Welcome to WebmasterWorld [webmasterworld.com]!

The location of your log files would depend on how your hosting service has your account configured. I would recoomend that you check their FAQs, or contact their customer service department.

Jim

7:29 am on Mar 2, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 24, 2003
posts:125
votes: 0


Googlebot just hit me a few min ago...
Freshbot?
64.68.82.39
64.68.82.18
64.68.82.5
7:33 am on Mar 2, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Oct 23, 2002
posts:1052
votes: 0


Welcome to WebmasterWorld, markusf.

Meet one of our Google experts and read one of his classics: [webmasterworld.com...]

4:52 pm on Mar 2, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Dec 6, 2002
posts:279
votes: 0


I think deepbot finished yesterday (March 1). Bring on the update!
8:11 pm on Mar 2, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 5, 2002
posts:529
votes: 0


Ive had 850 pages indexed by googlebot THIS month already!

Come on update! :-/

10:58 pm on Mar 2, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 20, 2002
posts:139
votes: 0


I have not noticed the deep crawler since Feb 16. Today I've had a ton of traffic from the freshbot though, spidering a huge chunk of my sites.
12:10 am on Mar 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Aug 26, 2002
posts:122
votes: 0


And still going,

I got about 76 hit from googlebot today.

12:21 am on Mar 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 2, 2003
posts:41
votes: 0


During this deepcrawl GoogleBot had indexed over 60,000 pages over a period of 5 days. Then a few days of no activity. Then another 500 pages. A few days more of inactivity. Then last night an additional 3000 pages indexed. Very strange activity.
1:59 am on Mar 4, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:807
votes: 0


:::Then last night an additional 3000 pages indexed.

Is he showing up under 216.239.46.** or 64.68.82.**? The deepcrawl ended on about the 15th of last month.

216 is deepcrawl and 64 is the freshbot.

2:15 am on Mar 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 2, 2003
posts:41
votes: 0


My logs roll over every hour since they get large very quickly. I was unable to find out what bot was crawling last night. Wouldn't It would be strange for freshbot to crawl that many pages?
2:18 am on Mar 4, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 17, 2003
posts:103
votes: 0


Has the deep crawl happened yet. I have not seen google spidering my site and I have been watching like a Hawk? How long will it be before I see the updates from last months deep crawl?
2:25 am on Mar 4, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:807
votes: 0


Dppecrawl doesn't start until after the dance ends and lasts about 10 days.

:::Wouldn't It would be strange for freshbot to crawl that many pages?

Nope, not at all. Some of us get thousands of hits a day from the freshbot.

:::How long will it be before I see the updates from last months deep crawl?

When we start barking up the wall then you can be sure you can see them, er wait we are allready barking up the wall! The next dance will show the updates from last months deepcrawl.

4:52 am on Mar 4, 2003 (gmt 0)

New User

10+ Year Member

joined:Feb 7, 2003
posts:10
votes: 0


I, too, got hit by freshbot for over 50k pages.
First time that many pages in one day.

No sign of deepbot this month.

The funny thing is that freshbot is crawling
ten's of thousands of pages that are disallowed
in the robots.txt file. Anyone else seen this
happen recently?