Forum Moderators: Robert Charlton & goodroi
My homepage is hosted at a local provider here in Munich and has had pretty regular rates of traffic in the past (between 6.000 and 10.000
visits and about 1 GB of data transfer per month).
In May, traffic suddenly began to double every month and reached 44 GB in July. According to my logfile the increase was completely due to crawlers
originating from the domain "googlebot.com" which were pulling off loads and loads of Kilobytes.
I only noticed because my provider started billing me for additional traffic over and above my contractual limit of 5 GB a month. In July, I had to pay about
$150 extra.
As a journalist, I smelled a story here ("Google Costs Hapless Homepage Owners Millions"), so I contaced the German press office. As usual (Google is
rather paranoid in its dealings with the media) I received no reply, but immediately the invasion ceased!
Does anybody have a good explanation, or have you suffered similar invasions of googlebots yourself?
[edited by: tedster at 3:20 pm (utc) on Sep. 8, 2007]
[edit reason] removed specific url [/edit]
You did the right thing in reporting the problem. Google has a Help page about how to deal with crawling issues:
[google.com...]
A webmaster Tools account can be an asset in resolving all kinds of crawling issues.
.... you suffered similar invasions of googlebots yourself?
Yes. Twice in the last several years.
On both occasions I emailed Google and told them to behave or be banned.
In both cases they wrote back, apologized, and fixed their offending behavior.
The sites continue to have their usual high rankings for their important keywords in Google.
