Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: open
Did anybody see something like that in the logs? I tried a search here at WebmasterWorld and didn't turn up anything. Nor did I see it before in our logs.
This happened a day after a full crawl of our sites, although we have the respective *.js in place since the beginning of the year.
Im curious about what 'Googlebot/Test' is up for.
It does seem to obey the robots.txt file.
So because a rarely-seen spider didn't pull your particular .js files, you assume it obeys robots.txt?
Requests just one page, same one all 5 times.
only js in the file is a third party counter.
This is a very old page that hasn't been changed/updated in over a year except for site wide navigation changes.
GG did say he wasn't familiar with this bot and was going to check it out, but AFAIK he hasn't mentioned it again.
But why would the search engine care if one affiliate steals commissions from another one? Wouldn't they say, "hahha, serves them right!".
Just my thoughts
Would someone please teach this bot some manners!
If you want to find out which is happening, just place a few dummy functions in your js files. In those dummy functions open a window using an url that does not exist made up of two strings.
It might be necessary, to be certain, to connect the dummy functions to a small links somewhere on the page.
Hopefully, someone will be able to provide an answer rather than just speculate and ruminate.
[edited by: Powdork at 4:37 pm (utc) on Mar. 20, 2004]
Does this mean the pages will now be spidered with this new bot?
Enough reasons for me to say that you're either wrong or simply just trying to blow another bubble.
Regarding GBot/Test ignoring robots.txt: it wouldn't surprise me. Doh! A bot that doesn't identify itself with a user agent that contains correct contact info and/or correct whois info +plus+ GoogleGuy claiming that he doesn't know anything about a Testbot ... sorry, but this stinks ...
If you've read many of my posts you would know that I very rarely make definitive statements. When I do, you can bet your life that I can back them up.
When you have read and verified the stickymail I am about to send you, I trust you will apologise for calling me a liar.
There are a number of reasons both of you could experience what you have experienced and neither one would be wrong.
Google has made some MAJOR changes in the last month or so in the links they crawl and index. Probably the biggest change in two years from what I monitor.