Forum Moderators: open
Mediapartners-Google/2.1 (+http://www.googlebot.com/bot.html)
coming from Google IPs:
64.68.87.66
64.68.87.69
64.68.86.194
64.68.87.28
Hammered my server with 4-5 requests/per second per IP in a 10 minute period (monitoring stuff on my application cut them off).
Jeez. I thought Google bots were supposed to be well-behaved?! I've had over 3700 hits from Google bots since midnight.
Mike
"In regards to the "Media Partners" bot, Google does not maintain or operate this robot. The user agent for Google is "Googlebot"."
I wonder if someone is spoofing the Google UA and IP addresses?
But what for? If you spoof an IP address in a request, you can't get the reply back can you? And if not, isn't this basically amounting to a denial of service attack?
Mike
The IP is almost the range from Google.
We have the range 64.68.86.xx with some IPs in it, but not the
64.68.86.194, and we havnīt the IPs 64.68.87.xx at all in our list.
Even though thatīs nearby the Google-IP itīs not Google himself.
Was it crawling or requesting specific files?
If you spoof an IP address in a request, you can't get the reply back can you? And if not, isn't this basically amounting to a denial of service attack?
As far as I know in order to establish a connection the handshake needs to be completed - an IP spoof on it's own can't normally do this because the attacker can't "see" the data as it attempts to return to the spoofed source.
In this scenario IP spoofing only allows the attacker to attempt to open a connection but not complete the connection so they would be unable to actually make a request against the webserver.
However if the attacker was able to see that traffic on the way out then it would be possible to complete the handshake while still spoofing the IP - normally this requires the routing data to be corrupted (so the IP can be routed to the attacker's choice of destination) or for the attacker to be able to sniff the data as it back to the spoofed IP.
In the grand scheme of things this seems like a lot of work for just spidering websites while pretending to be Google (ARIN seems to think thats a Google IP)...
IMO it's more likely to be a new project, pet project or possibly even an open proxy within google (I seem to remember people saying these do happen from time to time).
- Tony
IP Range 64.68.80.0 - 64.68.87.255
I would still like to know and understand:
1. What is this bot doing that is different from other bots?
2. Does it obey robots.txt?
3. Can Google slow it down so it isn't hammering a server with several requests/second?
GoogleGuy, can you give us a definitive explanation of what this bot is about and why it has to move so fast?
Thanks,
Mike
It did over 10,000 accesses yesterday and 4800 today.
But they still would not explain what this particular bot is supposed to be doing.
GoogleGuy, can you tell us about this thing?
Mike
Fastclick, used on my site to serve ads, is serving ads for Google. When someone does a search on my site (pretty much all my site does), the query is being sent to Google to see if Google has an ad to serve for that keyword. If they do, Fastclick serves the ad.
Part of what Google gets out the deal, other than ad revenue, is a new page to spider. A URL that came from a media partner, hence the name of the bot.
So, Fastclick is providing Google a big fat pool of URLs to spider. This seems to me to be very clever.
Apparently this is so new that the mainstream bot team at Google hadn't heard of it yet, and the team managing the bot didn't understand how aggressively the bot was tearing through websites.
Anyone think this explanation is possible?
Mike
Anyone think this explanation is possible?
Yes it's possible but it's more lightly that each time a Google ad is shown on your site, Google crawls to determine the theme of that particular page. This would allow relevant CPC ads to be shown in the ad.
Curiously, I haven't see a Google Mediapartners bot in the past few days. Maybe they're caching the URLs for spidering later?
Google is running dynamic ads (based on Adwords) through FastClick and other ad providers. Just as- with adwords- relevent ads appear on your search results page based on the key words searched... when this ad appears on your site, this bot hits your site, looks for keywords on YOUR site, then sends a dynamic adwords ad based on the keywords on YOUR PAGE!
Make sense?
As soon as I saw this sort of ad from Google, I blocked it. I see ad words in DIRECT COMPETITION with me... and is loosing a sale from my site worth the 50 cents google will pay me. Heck no!
Ban it!
dave
I have seen common Googlebot visiting my other site with the same url as the Adwords. I add &source=adword and GB fetched that page. But everytime got 404. This is a valid page, works well from Adwords and I am getting visitors with this URL. But evertime GB tries to get it, it get 404. Next thing I know is an email from Adwords team saying that the URL not working.
A bit strange...
