Got hammered by Googlebot last night

[jimmykav]

Any one else see a large feast of html served up to her?

[Weblamer]

yup, came and gobbled my site in the wee hours of the morning.

[dazz]

Is it me? or does this make anyone else ''''CrInGe!''''

[Rugles]

We have seen deep crawling every day for over a week.

[WebRookie]

Yep, deep crawl in the early morning hours.

[Dino_M]

I know I'm not upto speed but how do you know when Google has crawled your site? Explaination in detail pls or point to relevant thread would be much appreciated.

[hskfun]

Looks like the deep crawl started on the 26th for me.
50k urls and counting...

[europeforvisitors]

I know I'm not upto speed but how do you know when Google has crawled your site? Explaination in detail pls or point to relevant thread would be much appreciated.

The information should be in your daily server logs. Many (most?) log-analysis programs have an entry for robots and spiders where you'll find entries for various "googlebots" and other search-engine spiders.

[WebGuerrilla]

It will look like this:

"GET /yourpage.html HTTP/1.0" 200 12101 "-" "Googlebot/2.1 (+http://www.googlebot.com/bot.html)"

If you are looking at a raw log file, you can use the find fubction included in most text editors and do a search for googlebot, or a specific IP range.

[windiablo]

I thought that the recent crawls, and updates, were over...but it's started again...any particular reason for this ?

Right now...one of my sites is invaded by 216.239.46...

[Weblamer]

Heh, i know when google crawls my page cuz i have a piece of asp code on my page so that whenever soemoen goes to look at a particular product code, their IP and the product code and the time gets emailed to me. On my end i wrote a special program in VB that receives this information and dumpes it into an access database. Its cool cuz i see exactly what was crawled and when it was crawled. THen i can take the product keyword and go to google to see how well it was indexed. :)

[Night_Hawk]

I found the last 10 visitor in my web hosting log was google 216.239.46... and 64.68.82..., but the funny thing I am running an asp code to record the ip addresses, the page visited, the keyword used for the search and where they came from (the referral site or SE) and there is no trace to google.!

Checking www, www2, www3 they all have the same pages (last month update pages).
Does the googlebot ip get recorded when it visit a site or does it use a backdoor if that is the right term to use.

Thanks for the help.

By the way, I am a new user as you can tell, but I been reading this forum for a few months before I signed up - My first site made a PR6 the first crawl and now I am PR5 and I owe it all to you.

[zeus]

Googelbot has been on my site all day, every 2 min and some times a break, but the spider is in action.

zeus

[Slade]

I've got GoogleBot hits this morning on a PR3 site.

[stimpy]

What happened on the 23rd?

Running a link:www.domainname.com on google was producing 136 pages coming in (on one of my clients sites) - in july and mid august now I got :

Your search - link:www.domainname.com - did not match any documents.

No one is submitting the site regular, but it still has 500+ pages indexed on google. No one is running heavy, or even moderate WPG rankings on it.

My top 10 rankings out of 371,000 serps (on Google) dropped to 90's+ And my client is crying that the phone stopped ringing... Im getting PR3 on the home these days - I dont know prior -

It seems like Google did a partial rebuild or something -

what happened? - Do I have to wait until Sept 20 for the next rebuild?
Asside from the PR ratings - how can I check if I messed something up?

[brotherhood of LAN]

I've had 4 times as many hits from the bot as usual. No increase in PR, in fact, PR went down for some inner pages last update.

Makes me wonder. Because I've been getting more traffic since the Summer hols finished. G doesn't track click through all the time no? No toolbar info? :)

Must be a coincidence. No extra content.

Nonetheless hope it means I'm listed next month!

It's been fairly constant last few days, and the "other" crawl, mid month or thereabouts, and a few weeks before that..... :)

[kris]

Got hammered by Googlebot last night!

Anybody else find that title mildly amusing!?

[PaulPaul]

nice one Kris

[zeus]

Well my site got a PR0 at this update with no reason, but the googlebot is still spidering my site also today.

zeus

P.s It has not been my week just got a soccer(football)injury and now 7 month no football.

[jimmykav]

Intended to be Kris, intended to be...