Forum Moderators: DixonJones
Arianna asked this back in January [webmasterworld.com], but seems not to have received a reply.
Investigate the IP address(es) you see it coming from, and observe its behaviour on your site. Does it fetch images? Custom error pages? Robots.txt disallowed resources? Does it fetch lots of resources all at once, or go straight for your "contact" information?
Jim
It was this lack of comment that made me wonder if it was legit, despite the fact that its name smacks of it hiding something.
If anyone has any information, it'll save me (and possibly others) a lot of time in following the line of enquiry suggested by jdMorgan if it gets posted here. Thanks in anticipation.
Also, I'm seeing an even greater number of reqs from "Netscape (compatible)" which looks to be different than the Netscape browser
In analog 5 currently, it is registering the yahoo slurp bot as 'netscape compatible', it was correctly id'ing the yahoo vertical crawler as such, but it seems like slurp is not being correctly id'ed.
This is one among many reasons people are incorrectly reporting inflated netscape/mozilla numbers, that one took me a while to realize, but a quick check of my logs this week showed many slurp nua strings, but none reported on analog 5, but the 'netscape compatible' accounts exactly for slurp visits.
Simple common sense tells me that 10-15% of the market is not using mozilla browsers, I think many of the mozilla/netscape compatible counts are counting spiders using 'mozilla/5' or 'netscape compatible' in their user agent strings.
This is a rough guess, but I think it's around this now on average, but a quick look at amazon/ebay stats for the last few months would instantly give a very accurate representation of current worldwide browser useage:
3% all mac browsers
5-8% all mozillas including netscape 4x
1% operas
the rest IE
Also, in your stats, make sure to take a very close look at the mozilla stats themselves, before I installed php spider blocking scripts on many of my sites, I used to get numbers like this:
Mozilla 1 1500
-mozilla 1.6 168
-mozilla 1.2 21
-mozilla 1.0 2
Mozilla 0.9 23
-mozilla 0.9.4 23
In other words, the spiders were being id'ed as mozilla 1, but without the subversion, since they weren't using rv:1.x in their navigator user agents, once I installed spider traps these numbers now reflect my actual mozilla visitors, in other words, the overall mozilla 1 count equals the subversion counts exactly, before it never did.