Welcome to WebmasterWorld Guest from 54.237.251.98

Forum Moderators: open

Message Too Old, No Replies

Is the crawl done?

No hits from Deep Crawler in 4 hours.

     
12:48 pm on Mar 18, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 30, 2002
posts:437
votes: 0


Should I be worried? GoogleBot (deep) has not hit any of my 5 sites in about 4 hours. My major concern is Google has gotten about half the number of pages so far this crawl as last crawl.

It started very slowly this month, and didn't start getting serious until a few days into the crawl.

Am I alone in GoogleBot being gone? Is it done?

-Pete

12:54 pm on Mar 18, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 22, 2003
posts:1022
votes: 0


Nope still kicking around here. Been a little slower than in the past though!

-gs

12:55 pm on Mar 18, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 5, 2003
posts:86
votes: 0



I have the exact same experience!
3:19 pm on Mar 18, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 3, 2003
posts:58
votes: 0


After a couple of intense days during which the Googlebots crawled about 55000 pages (which is considerably less than ususal), things went calm about 4am this morning. No sign of the Deep Crawlers since that time. So if the Deep Crawl is over, it's been a much less extensive one for me than the last couple of months...
4:20 pm on Mar 18, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 22, 2003
posts:134
votes: 0


Hi,

I have been experiencing the same. Since, the major deep crawl on March 10th, my sites have not been deep crawled even once. The freshbot is also slow.

5:11 pm on Mar 18, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:807
votes: 0


I think I scared him away this morning!
5:14 pm on Mar 18, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 10, 2003
posts:157
votes: 0


I saw about 25000 hits a day for a week, but today only the freshbot. Funny thing is - I only have 60000 some odd pages so it would seem deepbot crawed my entire site more than once (although I have not confirmed that...)
5:53 pm on Mar 18, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 22, 2003
posts:1483
votes: 0


Only slow freshbot since 3AM here too. Is he bringing up the rear for Deepbot? Get back to work Deepbot, you're not done.
6:53 pm on Mar 18, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 5, 2003
posts:86
votes: 0



Deepbot is back!
8:00 pm on Mar 18, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 3, 2003
posts:58
votes: 0


Yes, the Deep Crawlers have returned to my site as well. I'd be interested to know whether the Deep Crawl takes a full-on intermission or whether it simply moves on to other sites for a time. In any case, I'm pleased to have it back in the house...
8:00 pm on Mar 18, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 15, 2000
posts:482
votes: 0


coukd it be that you are a bit paranoid? ;-)

itīs the end of the spider cycle, so only tomorrow and maybe the day after tomorrow the deepbot is on the way. then itīs over for a few weeks.

9:11 pm on Mar 18, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 30, 2002
posts:437
votes: 0


Back for me as well. (I started this thread.)

Anyone know this to have happened before? It is obviously more than just me, as other people saw it stop, and come back more or less on all sites at once.

Hopefully it still has time to get the other half of the pages it usuallty gets.

-Pete

10:03 pm on Mar 18, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:June 3, 2002
posts:142
votes: 0


Still here...
4:01 am on Mar 19, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Mar 7, 2003
posts:90
votes: 0


Well wouldn't depend on how the google database is designed? and how the bots search - is it by domain name or ip address of the server, or the number of pages he wants to look at?

On my server I've seen an almost perfect bell curve starting 3/11 peaking 3/15 and decling down today 3/18

4:55 am on Mar 19, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 7, 2003
posts:167
votes: 0


IMHO DeepCrawl is not done just yet. Still some pages being spidered. It did take a break though which was noticeable.

And just to scare people. I have one site that was deep crawled back in December and has never been deep crawled again. Don't ask me why. The PRs are still fine. Freshbot appears from time to time, but deepbot is not there.

Sometimes we webmasters panic a little too readily.

7:09 am on Mar 19, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Aug 15, 2000
posts:482
votes: 0


normally the deepcrawl is separated into to sections with a little break between them. (break doesnīt mean, that it doesnīt come at all - it only takes a very few pages during this time) the first part takes about 5 to 6 days, the second one 3 to 5.
11:39 am on Mar 19, 2003 (gmt 0)

New User

10+ Year Member

joined:Feb 9, 2003
posts:24
votes: 0


Googlebot is crawling my server since march 13th and it is still there. Donīt stop babyyy :)
12:31 pm on Mar 19, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Oct 18, 2002
posts:145
votes: 0


DeepBot still here.

Btw...how long a deepcrawl usually go on?

thx

fabriZio

12:51 am on Mar 20, 2003 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 5, 2003
posts:807
votes: 0


10 days
2:22 am on Mar 20, 2003 (gmt 0)

New User

10+ Year Member

joined:Feb 12, 2003
posts:30
votes: 0


i dont get why the deepcrawl doesnt hit all of the pages its supposed to hit. any simple explainations or ways to get it, maybe its beyond our control?
4:58 pm on Mar 20, 2003 (gmt 0)

Preferred Member

10+ Year Member

joined:Jan 9, 2003
posts:359
votes: 0



Same there. Deepcrawl was constantly grabbing pages then it abruptly stop and does not come back.

Something wrong. Something to do with the war or what?

kwngian

5:42 pm on Mar 20, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Jan 3, 2003
posts:58
votes: 0


Since I first posted to this thread a couple of days ago, the Deep Cralers have been coming and going with no real pattern. There have been hours where I served them more than 1000 pages and others where it's been less than 50. It's not idea but so long as it continues I'll happily endure it...