Welcome to WebmasterWorld Guest from 54.205.96.97

Forum Moderators: open

Is the crawl done?

No hits from Deep Crawler in 4 hours.

   
12:48 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



Should I be worried? GoogleBot (deep) has not hit any of my 5 sites in about 4 hours. My major concern is Google has gotten about half the number of pages so far this crawl as last crawl.

It started very slowly this month, and didn't start getting serious until a few days into the crawl.

Am I alone in GoogleBot being gone? Is it done?

-Pete

12:54 pm on Mar 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Nope still kicking around here. Been a little slower than in the past though!

-gs

12:55 pm on Mar 18, 2003 (gmt 0)

10+ Year Member




I have the exact same experience!
3:19 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



After a couple of intense days during which the Googlebots crawled about 55000 pages (which is considerably less than ususal), things went calm about 4am this morning. No sign of the Deep Crawlers since that time. So if the Deep Crawl is over, it's been a much less extensive one for me than the last couple of months...
4:20 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



Hi,

I have been experiencing the same. Since, the major deep crawl on March 10th, my sites have not been deep crawled even once. The freshbot is also slow.

5:11 pm on Mar 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I think I scared him away this morning!
5:14 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



I saw about 25000 hits a day for a week, but today only the freshbot. Funny thing is - I only have 60000 some odd pages so it would seem deepbot crawed my entire site more than once (although I have not confirmed that...)
5:53 pm on Mar 18, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Only slow freshbot since 3AM here too. Is he bringing up the rear for Deepbot? Get back to work Deepbot, you're not done.
6:53 pm on Mar 18, 2003 (gmt 0)

10+ Year Member




Deepbot is back!
8:00 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



Yes, the Deep Crawlers have returned to my site as well. I'd be interested to know whether the Deep Crawl takes a full-on intermission or whether it simply moves on to other sites for a time. In any case, I'm pleased to have it back in the house...
8:00 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



coukd it be that you are a bit paranoid? ;-)

itīs the end of the spider cycle, so only tomorrow and maybe the day after tomorrow the deepbot is on the way. then itīs over for a few weeks.

9:11 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



Back for me as well. (I started this thread.)

Anyone know this to have happened before? It is obviously more than just me, as other people saw it stop, and come back more or less on all sites at once.

Hopefully it still has time to get the other half of the pages it usuallty gets.

-Pete

10:03 pm on Mar 18, 2003 (gmt 0)

10+ Year Member



Still here...
4:01 am on Mar 19, 2003 (gmt 0)

10+ Year Member



Well wouldn't depend on how the google database is designed? and how the bots search - is it by domain name or ip address of the server, or the number of pages he wants to look at?

On my server I've seen an almost perfect bell curve starting 3/11 peaking 3/15 and decling down today 3/18

4:55 am on Mar 19, 2003 (gmt 0)

10+ Year Member



IMHO DeepCrawl is not done just yet. Still some pages being spidered. It did take a break though which was noticeable.

And just to scare people. I have one site that was deep crawled back in December and has never been deep crawled again. Don't ask me why. The PRs are still fine. Freshbot appears from time to time, but deepbot is not there.

Sometimes we webmasters panic a little too readily.

7:09 am on Mar 19, 2003 (gmt 0)

10+ Year Member



normally the deepcrawl is separated into to sections with a little break between them. (break doesnīt mean, that it doesnīt come at all - it only takes a very few pages during this time) the first part takes about 5 to 6 days, the second one 3 to 5.
11:39 am on Mar 19, 2003 (gmt 0)

10+ Year Member



Googlebot is crawling my server since march 13th and it is still there. Donīt stop babyyy :)
12:31 pm on Mar 19, 2003 (gmt 0)

10+ Year Member



DeepBot still here.

Btw...how long a deepcrawl usually go on?

thx

fabriZio

12:51 am on Mar 20, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



10 days
2:22 am on Mar 20, 2003 (gmt 0)

10+ Year Member



i dont get why the deepcrawl doesnt hit all of the pages its supposed to hit. any simple explainations or ways to get it, maybe its beyond our control?
4:58 pm on Mar 20, 2003 (gmt 0)

10+ Year Member




Same there. Deepcrawl was constantly grabbing pages then it abruptly stop and does not come back.

Something wrong. Something to do with the war or what?

kwngian

5:42 pm on Mar 20, 2003 (gmt 0)

10+ Year Member



Since I first posted to this thread a couple of days ago, the Deep Cralers have been coming and going with no real pattern. There have been hours where I served them more than 1000 pages and others where it's been less than 50. It's not idea but so long as it continues I'll happily endure it...
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month