Google bot living on my site...

[daunk]

I have one googlebot which literally lives on my site, its getting quite disconcerting. Due to my site having constantly updating information (forum) I guess it needs to spider each new page an ad shows up on. The googlebot has not left my site for 2 weeks now, I'm worried his parents may be looking for him :(

I'm not complaining because the ads i am getting are really targetted, but just commenting that is this one whole computer my site takes up or can each computer run several googlebots?

Incedently I hope it does update my google.com SERP listing because otherwise it will only have to bring along its friend to do that causing google more load/cpu and me more bandwidth cost.

Anyone else have the same resident google?

[universetoday]

I've got one too. It's reading thousands of pages a day.

[mcavill]

no but i'd like one ;)

[RTM Communications]

Yup, we also have the GoogleBot (note, the MediaPartners AdSense bot) practically living on our sites - especially those that are highly dynamic (forums) and reindexing pages as they change with new posts etc.

It creates a lot of traffic, but our hosting is pretty robust and the traffic isn't a big issue.

The targetting quality varies greatly given the dynamic content, as well as other factors such as geo and language targetting.

As for this bot changing your ranking in the SERPs, I doubt it - most of the threads here seem to concur that the AdSense bot does not impact SERPs.

On a side note, if you are running dynamic sites (such as forums) it's well worth looking into creating static URLs for the real GoogleBot. Despite what many say about G crawling dynamic sites, it will definitely crawl static URLs MUCH quicker.

Rob

[trillianjedi]

Remove your session ID's.......

TJ

[RTM Communications]

No session IDs at our site... at least there shouldn't be. But I suspect GB is simply recrawling our pages regularly. Hmmm... will have a detailled look at our raw server logs.

Rob

[Jenstar]

Logout of your forums, then check to see if session IDs show up for guests. I had one little line of session ID code left on my boards, but it only showed up for those who were not logged in. I checked everything while I was logged in and never noticed it until someone pointed it out a couple of weeks later. So bots were seeing session IDs when they visited, even though I thought differently ;)

[shrirch]

Logout and turn your cookies off.

[jjohnstn]

Yes, *definitely* make your pages static and crawlable.... use Apache's mod_rewrite or whatever you have to do. Make sure the actual links are crawlable. This may require some code re-writing, but it will be worth it. Make sure the pages are keyword-optimized for SE's and you're good to go.

Googlebot has set up camp on my site, as have Teoma and Alexa. Pages are already listed in Google, and hopefully will soon be in Teoma (anyone know the lag time before appearing in their index?)

[jonknee]

Don't worry about hogging the Google bot daunk. It's faster than light and can handle a hell of a lot more than one forum at a time :). And yes, it's running on way more than just a single CPU. I'd hate to pay Google's bandwidth bill though!