google bots

anyone else still getting indexed?

[o0_cops_0o]

I got loads of google bots which appear to be freshbots, the question is, is it common to have all these fresh bots a site at once? there all on diffrent pages

64.68.82.45 - crawler11.googlebot.com 00:06:36

64.68.82.68 - crawler12.googlebot.com 00:04:45

64.68.82.50 - crawler11.googlebot.com 00:02:54

64.68.82.52 - crawler11.googlebot.com 00:01:16

64.68.82.16 - crawler10.googlebot.com

[lazerzubb]

Yeah it happens often to a lot of sites.
Also if you want to learn more about them i suggest reading:
Google Updates and Everflux, the Monthly Mid-Cycle Changes [webmasterworld.com]
Googlebot: Deepbot and Freshbot FAQ and Information [webmasterworld.com]

[o0_cops_0o]

damn these things are on my site at once 24/7 and wont go away
64.68.82.56 - crawler11.googlebot.com 00:34:54
64.68.82.54 - crawler11.googlebot.com 00:26:56
64.68.82.41 - crawler11.googlebot.com 00:24:57
64.68.82.16 - crawler10.googlebot.com 00:24:15
64.68.82.34 - crawler11.googlebot.com 00:20:11
64.68.82.31 - crawler11.googlebot.com 00:14:53
64.68.82.51 - crawler11.googlebot.com 00:13:36
64.68.82.32 - crawler11.googlebot.com 00:10:25
64.68.82.44 - crawler11.googlebot.com 00:09:31
64.68.82.37 - crawler11.googlebot.com 00:09:19
64.68.82.50 - crawler11.googlebot.com 00:09:16
64.68.82.48 - crawler11.googlebot.com 00:08:14
64.68.82.43 - crawler11.googlebot.com 00:08:14
64.68.82.55 - crawler11.googlebot.com 00:07:31
64.68.82.46 - crawler11.googlebot.com 00:06:45
64.68.82.71 - crawler12.googlebot.com 00:02:35
64.68.82.52 - crawler11.googlebot.com 00:00:36

[galaga]

I've got crawler10.googlebot.com, crawler11.googlebot.com and crawler12.googlebot.com on my site 24/7! :(
I assume this is normal, but does anyone know how long they will stay for?

[o0_cops_0o]

galaga

freshbots on the site is a good thing. im just having fun watching them come and go :) its a new site and im just excited. now i have aol on there :D
64.12.96.201 America Online, Inc. US,VA cache-mtc-ak04.proxy.aol.com 00:04:40

64.12.96.8 America Online, Inc. US,VA cache-mtc-aa03.proxy.aol.com 00:03:59

64.12.96.205 America Online, Inc. US,VA cache-mtc-ak08.proxy.aol.com 00:03:57

64.12.97.13 America Online, Inc. US,VA cache-mtc-am08.proxy.aol.com 00:03:56

64.12.96.202 America Online, Inc. US,VA cache-mtc-ak05.proxy.aol.com 00:03:56

64.12.96.42 America Online, Inc. US,VA cache-mtc-ab05.proxy.aol.com 00:03:56

64.12.96.138 America Online, Inc. US,VA cache-mtc-ag05.proxy.aol.com 00:03:56

64.12.96.238 America Online, Inc. US,VA cache-mtc-al09.proxy.aol.com 00:03:48

64.12.96.198 America Online, Inc. US,VA cache-mtc-ak01.proxy.aol.com 00:03:40

64.12.96.237 America Online, Inc. US,VA cache-mtc-al08.proxy.aol.com 00:00:29

[galaga]

This is the first time I've updated my site for a while and it's been a pretty major update. I'm glad that the freshbots are a good sign, but I was wondering- what's the job of a freshbot? I mean, how does it differ from other bots? Because I thought there were just lots of little googlebots zooming around indexing all the web in the same way. I was also wondering if it would be foolish for me to update my site (minor changes across most pages) while the bots are on my site, or should I wait until the crawl is finished?

[galaga]

Okay, just found the faq thread about bots, so I get it now. However(!), I uploaded my updated site on the 2nd Feb, for the week prior to this, it was the old version of the site and before that the site was down for a roughly a month. So I'm wondering, have I missed the deep crawl and if so, what will happen if I missed the deep crawl but am getting hit by freshbots that discover a redically updated site? Does the site stand much chance of being included in the update? And if it doesn't, will it stand a good chance of appearing in the next one? I'm also still pondering on the wisdom of updating a wee bit more at the moment?! Sorry for the barage of questions, but a webmaster can get a bit twitchy when being ravaged by strange bots!

[jdMorgan]

galaga,

I uploaded my updated site on the 2nd Feb

February 2nd was before the deep crawl started; 99.9% chance you'll be in the new index. :)

Jim

[galaga]

Excellent :D But I'm still wondering if I can update any of my pages or if I should wait until after the update?

[peterdaly]

Excellent :D But I'm still wondering if I can update any of my pages or if I should wait until after the update?

I believe the deep crawl is over. Google has everything it is going to get into the deep index this cycle. Only reason to not keep working on your site would be to make sure users get exactly what Google saw. I personally think it makes more sense to keep working on the site, just keep the effect of the google crawl in mind when making drastic changes.

-Pete

[Jesse_Smith]

I think any one who ONLY updates there site between the time the dance ends and the deeppcrawl starts, about only five days, has gone nuts. If you start making changes now then you will have a lot more updated stuff for the next deepcrawl. If you wait until after the dance then you will only have about five days to update and add new pages and stuff.

[ciml]

galaga, you can update your pages, but the changes might not show until the update after next.

[galaga]

thanks guys!