Forum Moderators: open

Message Too Old, No Replies

Crawl behavior from 64.68.82.xxx

Deep bot back?

         

Clark

10:38 am on Jun 29, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Check your logs. I believe the first google deepcrawl in 2 months has started. I'm seeing pages indexed that have never been indexed before. I'm not 100% sure, but 80%. Please confirm

KDE_Head

2:44 pm on Jul 3, 2003 (gmt 0)

10+ Year Member



getting a seriously deep crawl since 3AM today and its still going.

interestingly , no Googlebot entries on July 1st whatsoever (calm before the storm?) - crawl starts July 2nd, with a serious escalation today. I'm getting several hits per minute , with pauses for 20 minutes or so. Multiple 64.68.82 bots.

On my other site, no freshbot entries in today's log whatsoever -maybe this is a sign of an eminent deep crawl?

Powdork

5:24 pm on Jul 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



KDE,
I'm watching the little buggers all over my site currently

farside847

5:39 pm on Jul 3, 2003 (gmt 0)

10+ Year Member



same here, hit every 10 seconds on average since 11PM last night...

markus007

6:42 pm on Jul 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



For the last month freshie has been grabbing 200 or so pages max from my site. Today its fetched 2500 pages + and counting.

Clark

6:42 pm on Jul 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



So far I've gotten half as much of a crawl as a few days ago, but this is one is still going strong...if it last for about 10 days, then we'll be talkin' :)

swampy webber

7:15 pm on Jul 3, 2003 (gmt 0)

10+ Year Member



When I got this behavior last weekend and I posted it, google visited several hundred pages. The little guy's at it again. The good news consists of two letters... fi

Has anyone checked out -fi ? Looks like my recently crawled data is being included.

Clark

7:19 pm on Jul 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



-fi didn't do anything for me, but glad it works for you...

Oaf357

7:50 pm on Jul 3, 2003 (gmt 0)

10+ Year Member



Just throwing in my $0.02 here but I'm seeing "Deepbot-like" behavior on my site as. New content getting crawled, deeply.

JoeHouse

12:12 am on Jul 4, 2003 (gmt 0)

10+ Year Member



What are we talking about here? Didn't the deep crawl occur on June 15 of last month? Are you guys trying to tell me its happening again or is this something different? I am confused. Please advise. Thanks!

Clark

3:55 am on Jul 4, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Joe,

I haven't heard of a June 15th deepcrawl. June 29 had a relatively deep crawl and today there was one of about the same size, ending (for me at least) at 3:30 pm.

Fearless

5:55 am on Jul 4, 2003 (gmt 0)

10+ Year Member



Newest site: crawl33.googlebot.com stopped by, asked for robots.txt and left

Still just one page indexed

Established sites: crawled deeply

akognet

4:03 pm on Jul 5, 2003 (gmt 0)

10+ Year Member



I'm seeing minimal hits/day since 3 July from 64.68.82 (none before), and they seem to be crawling the links found on a previous version of my index page, instead of the current one.

But all the pages that were crawled on 3 July are already included in the index... Hmmm...

markus007

11:43 pm on Jul 5, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



My pages that where crawled recieved fresh tags. New pages that where crawled where not added to the index. This is the same thing that happened in may after dominic...

kstprod

12:10 am on Jul 6, 2003 (gmt 0)

10+ Year Member



Pages being crawled with changes, yet no fresh tags and no changes. This is the 2nd crawl for me in a row like this. Hmmm is right.
This 44 message thread spans 2 pages: 44