Googlebot Spotted Deep Crawling

Update coming soon?

[Net_Wizard]

Active deep crawling activity early this morning March 3rd eastern time and been going on since this posting March 4 12:45 AM Eastern.

64.68.92.xx range Googlebot/2.1 (+http://www.googlebot.com/bot.html

I think this is it folks, everybody been waiting for :)

Perhaps a big update(including page rank) will follow.

Cheers

[phantombookman]

I don't pretend to know technically what that means but I hope the outcome is true.
Have put in loads of work thanks to what I have learnt on these boards in the past 2 months, still waiting to see the results.
Fingers crossed
Regards
Rod

[pendanticist]

You mean this?

64.68.82.143 - - [01/Mar/2004:20:11:03 -0800] "GET /robots.txt HTTP/1.0" 200 1565 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.143 - - [01/Mar/2004:20:11:03 -0800] "GET / HTTP/1.0" 200 - "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

Since the first of March I've had Twelve hits. Seven were for robots.txt, Four HTTP 1 and a lone 404 that'd been 301'd ages ago.

Unless you see something here that I don't.

[Net_Wizard]

Sorry guys, I've been out all day long...trying to make a living ;)

Anyway...I mean this 64.68.92.xx 'the whole ip range'.
Just check it again as I posted...'ALL DAY LONG' March 4 with UA "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

and again going deep including URL's with '?'

I lost count by now but I got hundreds of Googlebot access.

It won't be long it would in you site.

[markus007]

i have thousands of googlebots on my site per day, but they just seem to be validating old URL's, no real deep crawling going on.

[Net_Wizard]

I'm seeing some activity in some of my sites as well. Googlebot requesting some inner pages...nothing fast yet on my other sites about a few access per hour different pages and might escalate later on.

Am I the only one seeing this?

[Net_Wizard]

thousands of googlebots on my site per day

Well, lucky you ;)

But for some of us who have been seeing only 1 Googlebot a day or not even one this might be something else.

[PeterD]

May or may not be related: Anybody notice this new googlebot User-Agent string? I started seeing it March 2. Coming from legit googlebot IPs (crawler11 and crawler13):

"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

The old string was

"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

I also noted that the URL in the old string is being redirected to the URL in the new string.

(Forgive me if this has been mentioned before, but I couldn't find any mention of it.)

[internetheaven]

Yes, PeterD, someone has mentioned it a couple of days ago but no-one seemed to notice that the old URL now re-directs to the new one.
I've had a gut feeling that Google were doing an overhaul to their crawler since Florida. I figured that the low number of hits that everyone has experienced over the past month or so was due to testing of the new crawler. They wouldn't let a new crawler go as fast as the old one incase of bugs. Maybe testing is over and now the increase in hits is not an impending update (I certainly hope not anyway) but the new crawler being given more freedom to roam?

[Trisha]

Something seems to be going on, finally. I've had a new section of pages added to a site for a few weeks, and googlebot wouldn't crawl them. After reading this thread, I checked again, and now almost all of the pages have white bars!

[internetheaven]

Does anyone know of a Google Dance tool that still works? All of the ones I used to check with are now useless due to the IP changes.