deep crawling?

[icebane]

Has anyone been experience a lack of deep crawling the past week?

Thanks for any info.

[Miop]

Yes.

[Carswell]

Yes

[skodman]

Yes!

(just to join the club of very inventive answers)

[teeceo]

more like a "jump-ship" from the great googlebot. its beens super slow on my end.

[TinkyWinky]

no :(

[simonuk]

Yes but not just the past week. This has been happening on a number of my sites for the last few weeks (since around the time of Florida). Not all but some.

Simon.

[AjiNIMC]

I think yes but one of new site got crawled , i must say deeply crawled.
Aji

[superscript]

Woooah - now I'm confused ;) The question was

Has anyone been experience a lack of deep crawling

Which of the yes's mean no? And did the guy who wrote

No :(

mean

No :)

or

Yes :(

:-Q

[edited by: superscript at 12:43 pm (utc) on Dec. 23, 2003]

[skodman]

superscript, good point

i was asking my self same question.

skodman

[webdude]

I have seen very little activity for the G this past week. A few hits on robot.txt and the index page. Last deep crawl, I believe was last Thurday for me.

[nileshkurhade]

Googlebot registered no activity on my site ... But that has been the trend for a few weeks now ...

[TinkyWinky]

superscript - your damn right! Even more confused?

Too many do's and a pickled liver means I didn't read it properly,.....

I haven't had deep crawl - so it should be .....

Yes :(

[superscript]

Yes :(

LOL - now I'm really confused - oh s*d it, it's Christmas!

:)

[BroadProspect]

in the crawler alert service we are seeing almost no GoogleBot activity for the '000 of sites which we are monitoring, we do see fresh bot activity to the robots and main index pages but that is almost all.
/BP

[superscript]

Thanks BroadProspect - that's really interesting - we can forget all those yes, no's now ;)

<snip>

Added: for roughly how long has the activity level been so low?

[Silent_Bob]

I've noticed the same thing - very little googlebot activity for the last week or so. Maybe google have given googlebot a few weeks off for christmas?!

[marcus anthony]

Im New To WebMaster World so please excuse me if this is posted in the wrong place.
[Q]Our Site has been showing up in the Google index for a few months now, and the pages that are indexed change quite regularly, but its only the index page and some of the pages of the index page that ever get indexed.
For some unknown reason it won’t go deeper into the site can someone assist me in anyway?
[Q]Can someone clarify if I would be penalized for submitting my whole site to Google? (Including sections and subsections) As ive read on another post that this might be the answer.([Q]Why?
Google has spidered my site on several occasions but the site has never been deeply crawled.
[Q]Am I missing something?
The site is built using Actinic Catalog and uses CGI scripts on the index page, will this be having a negative affect or does this not make any difference.
Thanks in Advance

[BroadProspect]

I must say that it is only an assumption, but in the crawler alert service headqurters we are seeing that right before a deep crawl there is a small period where there is no crawl activity on any of the sites we are monitoring (since we monitor '000 of sites in europe, US and asia) we can detect that something is going on ..., so it started on friday before this weekend so we expected a deep crawl over the weekend (which seems to be a patten now, maybe because of the low commercial traffic over the weekend (and I do not mean adult oriented traffic ;-> ))
So our assumption right now is that google did plan to do a deep crawl this weekend but somehow it wasn't done, and the non googlebot activity is continuing) so either they plan to do it in the coming USA holiday (I'm from israel but I heard that there is one this week in the USA ) or over the coming weekend.
Again, just an assumption guys ...
/BroadProspect

[dwilson]

I launched a new site November 24th. It got hit almost right away, but only the index page & robots.txt.

Since then, only those two have been hit, though this last week they've been hit almost daily. And I'm seeing the fresh date updated every day or two in search results.

Can't wait for the deep crawl!

[Kwix]

No on the deep crawling. Also, has anybody else noticed Google waffleing on the cached version?

Right now, on www the cache of my homepage is from 12/15/03, three hours ago it was from 12/19/03. Old data serves what purpose?

[dwilson]

Right now, on www the cache of my homepage is from 12/15/03, three hours ago it was from 12/19/03. Old data serves what purpose?

www is actually served by multiple datacenters. Different ones may have different cached info.

[too much information]

I had a deep crawl recently, but none of my pages are showing up as being indexed.

I have probably 50 pages of good content, but only three show up as indexed. I believe the problem is that my site map was completed without commenting out the links to pages that are not yet complete. Since the site is designed from a template every page that is not complete is the same with the exception of the URL. (Major duplicate content)

I have fixed that, but I still can't understand why only 3 of the 50 completed pages show up as indexed, but not any others.

Maybe I set off a flag of some sort?

[cyberprosper]

Googlebot has been very quite for me too. I suspect they are using the extra horsepower available now to churn out a new and improved index. As I recall in the past, Googlebot slowed down substantially before a major update. Perhaps that is about to happen again.

[DRGather]

Deep crawl sometime last month and again on roughly the 10th (sorry, didn't note the date). All pages are indexed now, but no PR update as of yet. However it's been DEAD for about the last 3 or 4 days. No deep activity... hasn't even coming sniffing for robots.txt or the HP in the last several days. I'd agree on the lag in activity before a deep crawl in most cases. That's what happened to me on the 10th. Did anybody else get deep crawled earlier this month? (it may be of note that I'm a small fish in a big pond and my deep crawl lag interval is likely larger than some others around here.)

[superscript]

O.K. - here it comes - crazy off the wall theory - I was thinking about this last night ;)

Is it possible they are re-building their index from scratch? (on a data centre not visible) Producing a new map initially with index pages; then a deep crawl?

There's certainly been some strange behaviour recently: not just in the SERPs and data centres, but in the crawling patterns. And GG has essentially admitted the Directory was bust for a period. It's abundantly clear that some sites are going up and down like yo-yos.

x-mas joke courtesy of the BBC: "Two snowmen in a garden - one says to the other 'can you small carrots?'

[edited by: superscript at 4:56 pm (utc) on Dec. 23, 2003]

[werty]

No (c:

Googlebot hit a site I work on really hard last wed. night/thurs morning. The pages all got fresh tags...and the site was doing wonderful, then sat they dropped...no where to be be found now /c:

[ganderla]

I got 2 PR6 incomings on a brand new site on Thursday, almost every page is indexed now as there were no pages indexed this weekend.

[instabill]

No to the deep crawling - haven't seen googlebot beyond the index page since last thursday... Unusual as we usually see about 1K visits a day from the googlebot

[rfgdxm1]

What are you doing wrong? Googlebot not only can't stay away from my sites, but also from those of a few others by others I have SEOed. I assure you, Googlebot lives.