homepage Welcome to WebmasterWorld Guest from 54.196.199.46
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Visit PubCon.com
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Is the crawl done?
No hits from Deep Crawler in 4 hours.
peterdaly




msg:117423
 12:48 pm on Mar 18, 2003 (gmt 0)

Should I be worried? GoogleBot (deep) has not hit any of my 5 sites in about 4 hours. My major concern is Google has gotten about half the number of pages so far this crawl as last crawl.

It started very slowly this month, and didn't start getting serious until a few days into the crawl.

Am I alone in GoogleBot being gone? Is it done?

-Pete

 

Alternative Future




msg:117424
 12:54 pm on Mar 18, 2003 (gmt 0)

Nope still kicking around here. Been a little slower than in the past though!

-gs

leifwessman




msg:117425
 12:55 pm on Mar 18, 2003 (gmt 0)


I have the exact same experience!

uber_boy




msg:117426
 3:19 pm on Mar 18, 2003 (gmt 0)

After a couple of intense days during which the Googlebots crawled about 55000 pages (which is considerably less than ususal), things went calm about 4am this morning. No sign of the Deep Crawlers since that time. So if the Deep Crawl is over, it's been a much less extensive one for me than the last couple of months...

guddu




msg:117427
 4:20 pm on Mar 18, 2003 (gmt 0)

Hi,

I have been experiencing the same. Since, the major deep crawl on March 10th, my sites have not been deep crawled even once. The freshbot is also slow.

Jesse_Smith




msg:117428
 5:11 pm on Mar 18, 2003 (gmt 0)

I think I scared him away this morning!

farside847




msg:117429
 5:14 pm on Mar 18, 2003 (gmt 0)

I saw about 25000 hits a day for a week, but today only the freshbot. Funny thing is - I only have 60000 some odd pages so it would seem deepbot crawed my entire site more than once (although I have not confirmed that...)

skipfactor




msg:117430
 5:53 pm on Mar 18, 2003 (gmt 0)

Only slow freshbot since 3AM here too. Is he bringing up the rear for Deepbot? Get back to work Deepbot, you're not done.

leifwessman




msg:117431
 6:53 pm on Mar 18, 2003 (gmt 0)


Deepbot is back!

uber_boy




msg:117432
 8:00 pm on Mar 18, 2003 (gmt 0)

Yes, the Deep Crawlers have returned to my site as well. I'd be interested to know whether the Deep Crawl takes a full-on intermission or whether it simply moves on to other sites for a time. In any case, I'm pleased to have it back in the house...

oLeon




msg:117433
 8:00 pm on Mar 18, 2003 (gmt 0)

coukd it be that you are a bit paranoid? ;-)

itīs the end of the spider cycle, so only tomorrow and maybe the day after tomorrow the deepbot is on the way. then itīs over for a few weeks.

peterdaly




msg:117434
 9:11 pm on Mar 18, 2003 (gmt 0)

Back for me as well. (I started this thread.)

Anyone know this to have happened before? It is obviously more than just me, as other people saw it stop, and come back more or less on all sites at once.

Hopefully it still has time to get the other half of the pages it usuallty gets.

-Pete

KakenBetaal




msg:117435
 10:03 pm on Mar 18, 2003 (gmt 0)

Still here...

why2kit




msg:117436
 4:01 am on Mar 19, 2003 (gmt 0)

Well wouldn't depend on how the google database is designed? and how the bots search - is it by domain name or ip address of the server, or the number of pages he wants to look at?

On my server I've seen an almost perfect bell curve starting 3/11 peaking 3/15 and decling down today 3/18

MetropolisRobot




msg:117437
 4:55 am on Mar 19, 2003 (gmt 0)

IMHO DeepCrawl is not done just yet. Still some pages being spidered. It did take a break though which was noticeable.

And just to scare people. I have one site that was deep crawled back in December and has never been deep crawled again. Don't ask me why. The PRs are still fine. Freshbot appears from time to time, but deepbot is not there.

Sometimes we webmasters panic a little too readily.

oLeon




msg:117438
 7:09 am on Mar 19, 2003 (gmt 0)

normally the deepcrawl is separated into to sections with a little break between them. (break doesnīt mean, that it doesnīt come at all - it only takes a very few pages during this time) the first part takes about 5 to 6 days, the second one 3 to 5.

bignooz




msg:117439
 11:39 am on Mar 19, 2003 (gmt 0)

Googlebot is crawling my server since march 13th and it is still there. Donīt stop babyyy :)

joker197cinque




msg:117440
 12:31 pm on Mar 19, 2003 (gmt 0)

DeepBot still here.

Btw...how long a deepcrawl usually go on?

thx

fabriZio

Jesse_Smith




msg:117441
 12:51 am on Mar 20, 2003 (gmt 0)

10 days

Hefner




msg:117442
 2:22 am on Mar 20, 2003 (gmt 0)

i dont get why the deepcrawl doesnt hit all of the pages its supposed to hit. any simple explainations or ways to get it, maybe its beyond our control?

kwngian




msg:117443
 4:58 pm on Mar 20, 2003 (gmt 0)


Same there. Deepcrawl was constantly grabbing pages then it abruptly stop and does not come back.

Something wrong. Something to do with the war or what?

kwngian

uber_boy




msg:117444
 5:42 pm on Mar 20, 2003 (gmt 0)

Since I first posted to this thread a couple of days ago, the Deep Cralers have been coming and going with no real pattern. There have been hours where I served them more than 1000 pages and others where it's been less than 50. It's not idea but so long as it continues I'll happily endure it...

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved