Forum Moderators: open
I may be missing something, but I wonder how such a user agent:
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11"
can be visiting my site from IP address 66.249.84.10
in the Google.com net range.
Furthermore, this agent does not read robots.txt, falls in robotraps, submits hidden forms, request pages within an average of 4 sec., so looks more like a robot, but also executes Javascipts and supports cookies, which looks more like a browser.
Is Google also acting as a provider?
Any thought?
and these tools make requests from google server, using the user's browser identification?
An axample?
Ok, but if it is done "manually", how come they open robot traps and forms action pages with no submit button?
Pardon me for my septicism, but there is still something not correct from a company suposedly serious as Google:
Either they provide client side tools to users, as addon to their browser, but in that case the visitor will use his own provider's IP, either they provide tools to be used from their own server, but in that case it is not correct that their server mimics the visitor's browser.
Something is not clear here.
Something is not clear here.
In the event your not content with the activities of Google?
Simply go to ARIN
type in "google" in the Whois-Search box
Copy the results
and then add every range to your firewall.
>>some doAn axample?
search the archives.
C'mon, this is not the point.
I'm just trying to improve my robot evaluation module based on behaviour of robots.
I'm actually having problems with Google, which I do not want to tag as a bad robot of course, but the fact is that I get suspicious hits from an IP address in the google range, and I'm trying to understand why? That's all.
Forget it, I'll rather search for another forum with more cooperative people giving sensible answers or with a decent search engine, or both ;-)
Your search engine is just crap: it mostly returns pages in which the content has scrolled away :-(
Google (66.249.84.10), not Googlebot hit my site today while doing a "site:" search for a specific page URL. So, basically I'm assuming that this is a human review. They were checking to see if a certain URL was in their index or not. Funny thing is, that when I do the same search on my end the URL isn't indexed, but I guess the data center they are using shows the indexed URL, otherwise I wouldn't have seen the footprint in my analytics program.
Interesting :)
Other Google services:
Wireless Transcoder
Translation Service
If the real human is searching for a product unrelated to SE business, wouldn't they just search using the product part# as the keyword?
Why on earth would they do something like this?:
site:www.bluewidgets.com/blue-widgets-blah-blah-blah-specifications-blah-blah-blah.html
Unless of course, they are checking to see if that URL exists in Google's index.
