Googlebot is back!

It looks like the deep crawl is starting.

[RoadRash]

Google started to deep crawl my site in the last 15 minutes. Anyone else notice this?

[peterdaly]

6 "knocks" last night for me.

 agent   ¦ clientip ¦ count 
----------------------------------------------------+-----------------+------- 
 Googlebot/2.1 (+http://www.googlebot.com/bot.html) ¦ 216.239.46.140 ¦ 1 
 Googlebot/2.1 (+http://www.googlebot.com/bot.html) ¦ 216.239.46.173 ¦ 1 
 Googlebot/2.1 (+http://www.googlebot.com/bot.html) ¦ 216.239.46.193 ¦ 1 
 Googlebot/2.1 (+http://www.googlebot.com/bot.html) ¦ 216.239.46.55 ¦ 1 
 Googlebot/2.1 (+http://www.googlebot.com/bot.html) ¦ 216.239.46.63 ¦ 2 

Time to get any changes in....NOW!

-Pete

[nyehouse]

For clarification please.

The only issue with new content is new pages, or links, link text etc. relating to page rank.

There is no rush to change page copy or tags since freshbot can update these fields regularly?

Am I right, or does the deepbot (googlebot) keep its data in a separate index, so even if I change my entire page, the algo still works on the old cache copy?

Thanks

[Jesse_Smith]

It's back. Got six knocks yesterday evening, and just got three more hits 15 minutes appart. Another one of my sites got 13 hits in the last 15 minutes compaired to one hit yesterday! I think it's officially started bombing us!

[rharri]

I've got a small site (~150 pages; PR4) and the 216xxx googlebot has been pulling pages yesterday and today. I see at least 3 different IPs starting with 216.

[peterdaly]

It's back. Got six knocks yesterday evening, and just got three more hits 15 minutes appart. Another one of my sites got 13 hits in the last 15 minutes compaired to one hit yesterday! I think it's officially started bombing us!

Yup, another 67 hits since 14:44EST. 2/3 hits per minute. Interesting...my PR3 site is getting hit harder than my PR5 site. Maybe it's due for a PR upgrade this time around. :-)

-Pete

[xlcus]

*sob* My DNS went down 24 hours ago and I haven't had a single access from the DeepCrawl bot :-(

Strangely though, the FreshBot seems to have cached the ip of one (and only one) of my sites. Just a shame it isn't DeepCrawlBot :-( *sob*

[leifwessman]

My website has still not been spidered since jan 12. At that time I had some repeated keywords at the bottom of every page. The following week I removed all the keywords again. But that didn't help - my website (PR6) was still in the index but with the cache from jan 12.
Mars 2 i wrote to google about it ([e-mail]) but didn't get any response.

Some days ago I read GoogleGuys post that every website that was punished and have wrote to Google about it was going to be respidered again.

But why is my website still not spidered? It seems as if a lot of websites is getting spidered at the moment.

Leif

[yasunglass]

Check on www2.google.com or www3.google.com and see if you are there.

[srvdt]

Yesterday from 6:20 PM EST through midnight (I only get weblogs once a day - Booo!) I got hits from:

crawl1
crawl3
crawl4
crawl5
crawl6
crawl7

But they didn't go deeper than my top level dir. Is that pretty normal? Will they be back?

[creative craig]

I been hit hard on all sites by the deep crawler and the freshbot.

Craig

[uber_boy]

My site's PR7 and I've yet to receive any attention worth noting from the deep crawlers. I'm getting the usual intermittent burst of attention from FreshBot, but the deep crawl has not yet begun in earnest for me...

[johnraphone]

Googlebot has been visiting my sites. Several crawler4.googlebot.com, crawler9.googlebot.com, and crawler7.googlebot.com.

Good news for me, I updated my linking structure last night, I hope GoogleBot likes it. :)

[edited by: johnraphone at 11:04 pm (utc) on Mar. 11, 2003]

[bobmark]

"But why is my website still not spidered? It seems as if a lot of websites is getting spidered at the moment."

Well, actually half a dozen we know of from posts on here :)

I wouldn't worry too much yet...give it a few days.

Same for you, xlcus. The first appearance noted on here was last night so it's possible the deep crawl bot has not even visited your site yet. There was also a fairly broad router problem yesterday that prevented access from various locations to various locations so it may be Googlebot didn't even begin to deep crawl before last night.

[jady]

Launched a site, threw some links out there and got deep crawled all within 24 hrs.. What a machine! CHOMP CHOMP!

[sanity]

Yup deepbot's been a crawling here for the last 2 days.

[GoogleGuy]

I just have to point this out. My favorite GoogleBot fiction of all time (fiction, because of course a googlebot always obeys robots.txt):
[ftrain.com...]

I argued so hard that our Google Dance T-Shirts should have that GoogleBot on them. Oh well. The bot we got was cool too. But I love that little bot. :)

[rogue]

216. is eatin' me up. Hope she likes the whole meal. Have followed the recipe to the letter.

[DavidT]

Referring to Googlebot and repecting robots text, Freshbot has been having a good old go at a disallowed images folder on my site and been getting 403s for his trouble. Naughty, naughty.

Another thing, is Freshbot stupid? It can't seem to understand a 301 redirect from www.domain.com to domain.com. Keeps requesting pages and nothing but 301 in return.

[guddu]

Yes, Finally, the Googlebot is back.

I have seen two of my sites being deep crawled. Got 93 pages of one site and 20 pages of the other listed in Google. After a long wait, it was nice to see my pages listed in Google.

The Google Directory has been updated as well.

Hope, now the procedure would continue frequently.

[Alphawolf]

Yeah...GoogleBot likes me again. :)

After whining here about not seeing GoogleBot since Feb 28th- I got 2 FreshBot visits and 1 Deep Crawl visit just now.

I hope he will be back and bring some of his hungry friends.

So happy to contribute to this thread! ;)

AW

[ga_ga]

"I am Googlebot. Hello"

ROFLMAO

I am in pain with laughter

[spam90]

Looks like the deepcrawl spiders are out...

Just finished tweaking the site in time!

[TWhalen]

I'm seeing both fresh and deep crawlers right now.

[SubZeroGTS]

i'm only seeing freshbots so far (a lot of them).

surprisingly FAST/WebCrawler hit me up pretty deeply just a little while ago.

[o0_cops_0o]

after reading this post i checked my stats, deepcrawl on my site as well. I was uploading a bunch of pages i was working on while waiting for the update. You think this did something to my deepcrawl? guess i should wait till there gone to continue uploading?

[przero2]

deep crawl in full speed now.

[markusf]

Deep crawl here to...

[Rugles]

Here too.

I saw the deep crawler (216.xxx.xxx.xxx)very briefly this morning. Or it was someone from google actually surfing our site.

Now we have some serious crawling going on.

[ikbenhet1]

The deep crawl would start after the dance. So this means the dance finished?

[Jesse_Smith]

No, the dance isn't done. www-va and www-in still show last months resaults. But it looks like it almost is.