homepage Welcome to WebmasterWorld Guest from 54.226.161.112
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Googlebot Spotted Deep Crawling
Update coming soon?
Net_Wizard




msg:56394
 5:43 am on Mar 4, 2004 (gmt 0)

Active deep crawling activity early this morning March 3rd eastern time and been going on since this posting March 4 12:45 AM Eastern.

64.68.92.xx range Googlebot/2.1 (+http://www.googlebot.com/bot.html

I think this is it folks, everybody been waiting for :)

Perhaps a big update(including page rank) will follow.

Cheers

 

phantombookman




msg:56395
 7:06 am on Mar 4, 2004 (gmt 0)

I don't pretend to know technically what that means but I hope the outcome is true.
Have put in loads of work thanks to what I have learnt on these boards in the past 2 months, still waiting to see the results.
Fingers crossed
Regards
Rod

pendanticist




msg:56396
 7:06 am on Mar 4, 2004 (gmt 0)

You mean this?

64.68.82.143 - - [01/Mar/2004:20:11:03 -0800] "GET /robots.txt HTTP/1.0" 200 1565 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"
64.68.82.143 - - [01/Mar/2004:20:11:03 -0800] "GET / HTTP/1.0" 200 - "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

Since the first of March I've had Twelve hits. Seven were for robots.txt, Four HTTP 1 and a lone 404 that'd been 301'd ages ago.

Unless you see something here that I don't.

Net_Wizard




msg:56397
 8:01 pm on Mar 4, 2004 (gmt 0)

Sorry guys, I've been out all day long...trying to make a living ;)

Anyway...I mean this 64.68.92.xx 'the whole ip range'.
Just check it again as I posted...'ALL DAY LONG' March 4 with UA "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

and again going deep including URL's with '?'

I lost count by now but I got hundreds of Googlebot access.

It won't be long it would in you site.

markus007




msg:56398
 8:06 pm on Mar 4, 2004 (gmt 0)

i have thousands of googlebots on my site per day, but they just seem to be validating old URL's, no real deep crawling going on.

Net_Wizard




msg:56399
 8:12 pm on Mar 4, 2004 (gmt 0)

I'm seeing some activity in some of my sites as well. Googlebot requesting some inner pages...nothing fast yet on my other sites about a few access per hour different pages and might escalate later on.

Am I the only one seeing this?

Net_Wizard




msg:56400
 8:14 pm on Mar 4, 2004 (gmt 0)

thousands of googlebots on my site per day

Well, lucky you ;)

But for some of us who have been seeing only 1 Googlebot a day or not even one this might be something else.

PeterD




msg:56401
 8:26 pm on Mar 4, 2004 (gmt 0)

May or may not be related: Anybody notice this new googlebot User-Agent string? I started seeing it March 2. Coming from legit googlebot IPs (crawler11 and crawler13):

"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

The old string was

"Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

I also noted that the URL in the old string is being redirected to the URL in the new string.

(Forgive me if this has been mentioned before, but I couldn't find any mention of it.)

internetheaven




msg:56402
 9:32 pm on Mar 4, 2004 (gmt 0)

Yes, PeterD, someone has mentioned it a couple of days ago but no-one seemed to notice that the old URL now re-directs to the new one.
I've had a gut feeling that Google were doing an overhaul to their crawler since Florida. I figured that the low number of hits that everyone has experienced over the past month or so was due to testing of the new crawler. They wouldn't let a new crawler go as fast as the old one incase of bugs. Maybe testing is over and now the increase in hits is not an impending update (I certainly hope not anyway) but the new crawler being given more freedom to roam?

Trisha




msg:56403
 9:48 pm on Mar 4, 2004 (gmt 0)

Something seems to be going on, finally. I've had a new section of pages added to a site for a few weeks, and googlebot wouldn't crawl them. After reading this thread, I checked again, and now almost all of the pages have white bars!

internetheaven




msg:56404
 10:31 pm on Mar 4, 2004 (gmt 0)

Does anyone know of a Google Dance tool that still works? All of the ones I used to check with are now useless due to the IP changes.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved