homepage Welcome to WebmasterWorld Guest from 54.166.255.168
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Googlebot still crunching?
yum yum yum
AhmedF

10+ Year Member



 
Msg#: 9785 posted 7:27 pm on Mar 1, 2003 (gmt 0)

Hrmm ... Just checking my stats, and googlebot has visited my site 140 times in just 12 hours of this month ...

Is even the march 3rd guess too optimistic now?

 

AcesHigh



 
Msg#: 9785 posted 12:39 am on Mar 2, 2003 (gmt 0)

March 3rd STRONG believer here.

Dieter

10+ Year Member



 
Msg#: 9785 posted 3:39 am on Mar 2, 2003 (gmt 0)

Where do you check your stats to see that Googlebot has visited your site?

bhartzer

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9785 posted 3:56 am on Mar 2, 2003 (gmt 0)

Take a look at your log files for your site and you should be able to tell. Google disguises itself as 'googlebot'.

Dieter

10+ Year Member



 
Msg#: 9785 posted 4:06 am on Mar 2, 2003 (gmt 0)

Thanks bhartzer - where do I find the log files to my site? Is that the log from a Counter program, or is it something else?

jdMorgan

WebmasterWorld Senior Member jdmorgan us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 9785 posted 4:25 am on Mar 2, 2003 (gmt 0)

Dieter,

Welcome to WebmasterWorld [webmasterworld.com]!

The location of your log files would depend on how your hosting service has your account configured. I would recoomend that you check their FAQs, or contact their customer service department.

Jim

markusf



 
Msg#: 9785 posted 7:29 am on Mar 2, 2003 (gmt 0)

Googlebot just hit me a few min ago...
Freshbot?
64.68.82.39
64.68.82.18
64.68.82.5

troels nybo nielsen

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9785 posted 7:33 am on Mar 2, 2003 (gmt 0)

Welcome to WebmasterWorld, markusf.

Meet one of our Google experts and read one of his classics: [webmasterworld.com...]

webdevsf

10+ Year Member



 
Msg#: 9785 posted 4:52 pm on Mar 2, 2003 (gmt 0)

I think deepbot finished yesterday (March 1). Bring on the update!

AhmedF

10+ Year Member



 
Msg#: 9785 posted 8:11 pm on Mar 2, 2003 (gmt 0)

Ive had 850 pages indexed by googlebot THIS month already!

Come on update! :-/

Rick_M

10+ Year Member



 
Msg#: 9785 posted 10:58 pm on Mar 2, 2003 (gmt 0)

I have not noticed the deep crawler since Feb 16. Today I've had a ton of traffic from the freshbot though, spidering a huge chunk of my sites.

Night_Hawk

10+ Year Member



 
Msg#: 9785 posted 12:10 am on Mar 4, 2003 (gmt 0)

And still going,

I got about 76 hit from googlebot today.

DerekT

10+ Year Member



 
Msg#: 9785 posted 12:21 am on Mar 4, 2003 (gmt 0)

During this deepcrawl GoogleBot had indexed over 60,000 pages over a period of 5 days. Then a few days of no activity. Then another 500 pages. A few days more of inactivity. Then last night an additional 3000 pages indexed. Very strange activity.

Jesse_Smith

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9785 posted 1:59 am on Mar 4, 2003 (gmt 0)

:::Then last night an additional 3000 pages indexed.

Is he showing up under 216.239.46.** or 64.68.82.**? The deepcrawl ended on about the 15th of last month.

216 is deepcrawl and 64 is the freshbot.

DerekT

10+ Year Member



 
Msg#: 9785 posted 2:15 am on Mar 4, 2003 (gmt 0)

My logs roll over every hour since they get large very quickly. I was unable to find out what bot was crawling last night. Wouldn't It would be strange for freshbot to crawl that many pages?

kelleybelly

10+ Year Member



 
Msg#: 9785 posted 2:18 am on Mar 4, 2003 (gmt 0)

Has the deep crawl happened yet. I have not seen google spidering my site and I have been watching like a Hawk? How long will it be before I see the updates from last months deep crawl?

Jesse_Smith

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 9785 posted 2:25 am on Mar 4, 2003 (gmt 0)

Dppecrawl doesn't start until after the dance ends and lasts about 10 days.

:::Wouldn't It would be strange for freshbot to crawl that many pages?

Nope, not at all. Some of us get thousands of hits a day from the freshbot.

:::How long will it be before I see the updates from last months deep crawl?

When we start barking up the wall then you can be sure you can see them, er wait we are allready barking up the wall! The next dance will show the updates from last months deepcrawl.

wired4fun

10+ Year Member



 
Msg#: 9785 posted 4:52 am on Mar 4, 2003 (gmt 0)

I, too, got hit by freshbot for over 50k pages.
First time that many pages in one day.

No sign of deepbot this month.

The funny thing is that freshbot is crawling
ten's of thousands of pages that are disallowed
in the robots.txt file. Anyone else seen this
happen recently?

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved