Welcome to WebmasterWorld Guest from 54.205.75.60

Forum Moderators: open

Message Too Old, No Replies

Problem with class C Ip Checks

   
11:58 pm on Apr 6, 2002 (gmt 0)

10+ Year Member



Hi All, Question. If i take the ip address of the spider visiting, then knock off the end if it and make it a class c ip. ( 204.152.191.28 to 204.152.191) then do a search in the database it will flag that its a spider because of the folowing ips:204.152.191.41,204.152.191.47, 204.152.191.57, and 204.152.191.58. But Its altavistas translator service!
so whats the best way to get past this?

my pseudocode is:
function SpiderVerify(ip,UA)
'ip = ip address
'ua = User Agent
make ip class C
'123.123.123.123 to 123.123.123
check to see if any ips like that in database.
flag good for no matches.
flag bad for matches.

12:18 am on Apr 7, 2002 (gmt 0)

10+ Year Member



Also why make it a class C instead of the full IP?

Air

5:41 am on Apr 8, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The advantage is that spiders tend to be added from the same class "C" IP range, so it provides an opportunity to detect spiders that would otherwise be missed.
4:23 pm on Apr 8, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



You should keep a database of translators as well. Do a check on the translator database before the spider database. Here's the IPs I have listed as translators:

204.123.9.65
204.123.9.66
204.123.9.67
204.123.9.68
204.123.9.106
204.123.9.107
204.152.191.27
204.152.191.28
204.152.191.29
204.152.190.27
204.152.190.28
204.152.190.29
204.152.190.37
204.152.190.154
204.162.96.104
204.162.96.154
204.162.96.176
209.247.194.35
209.247.194.100
64.208.35.5

11:47 pm on Apr 8, 2002 (gmt 0)

10+ Year Member



Wow, Thanks so much, this is what i do:
first i grab the ip address and UA. then i also make the ip a class C. now what I do is check it against a sql2000 database and it does this check.
is IP a translator yes/no
if no
is class C ip like any spiders yes/no
if no
is UA like any spider UAs

This is as thurrow as i can get i think. Anyone else have any better ideas?

11:02 pm on Apr 9, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



I would do the check against the list of translators before truncating the IP address into a Class C.