Forum Moderators: open

Message Too Old, No Replies

Netscape/PICgrabber

         

keyplyr

12:08 am on Jul 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I don't have access to the IP because my weekly raw_logs are to large for me to open (RAM issue) but analog identifies Netscape/PICgrabber as getting 200 files and no pages.

How do I deal with the "/" when adding this to robots.txt?

Thanks

wilderness

10:38 pm on Jul 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



keyplyr
Just use it normally. "Netscape/PICgrabber"
However if you look at Jim's Robots Diet

[webmasterworld.com...]

The Standard is NOT using the slash or anything after it which would result in "Netscape"

Did this tool read robots? I kinda doubt it?
Your more likley going to need to resolve this in your htaccess.

SetEnvIf ^Netscape/PICgrabber keep_out (or what ever you use instead)

Don

jdMorgan

11:03 pm on Jul 2, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



We'll have to keep a lookout for the IP block on this one... I kind of doubt it's really a Netscape thing.

To clarify one point, I did not mean to imply that this user agent shouldn't be blocked with
Disallow: Netscape/PICgrabber

What I meant was, don't bother using something like
Disallow: NastyBot/1.2.3B
because then all they have to do is change the version number and come back, thus avoiding a technical violation. And, of course, trying to keep up with version numbers just increases your workload.

Jim

keyplyr

12:23 am on Jul 3, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Actually, I think it really is a Netscape tool, at least Analog thinks so since it's grouped in with the NN browser versions. Usually when a UA is spoofed, Analog just lists it individually.

Spoofed Internet Explorer or Googlebot gets listed by themselves, not in with the valid UAs. Many Opera users do this (for whatever reason?) so my stats for Opera aren't accurate; seems self defeating since they are always boasting how many users have switched to Opera, but that's OT ;)