Welcome to WebmasterWorld Guest from 23.20.147.6

Forum Moderators: open

Message Too Old, No Replies

Problem with class C Ip Checks

     
11:58 pm on Apr 6, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 3, 2002
posts:105
votes: 0


Hi All, Question. If i take the ip address of the spider visiting, then knock off the end if it and make it a class c ip. ( 204.152.191.28 to 204.152.191) then do a search in the database it will flag that its a spider because of the folowing ips:204.152.191.41,204.152.191.47, 204.152.191.57, and 204.152.191.58. But Its altavistas translator service!
so whats the best way to get past this?

my pseudocode is:
function SpiderVerify(ip,UA)
'ip = ip address
'ua = User Agent
make ip class C
'123.123.123.123 to 123.123.123
check to see if any ips like that in database.
flag good for no matches.
flag bad for matches.

12:18 am on Apr 7, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 3, 2002
posts:105
votes: 0


Also why make it a class C instead of the full IP?

Air

5:41 am on Apr 8, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Feb 10, 2000
posts:1253
votes: 0


The advantage is that spiders tend to be added from the same class "C" IP range, so it provides an opportunity to detect spiders that would otherwise be missed.
4:23 pm on Apr 8, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


You should keep a database of translators as well. Do a check on the translator database before the spider database. Here's the IPs I have listed as translators:

204.123.9.65
204.123.9.66
204.123.9.67
204.123.9.68
204.123.9.106
204.123.9.107
204.152.191.27
204.152.191.28
204.152.191.29
204.152.190.27
204.152.190.28
204.152.190.29
204.152.190.37
204.152.190.154
204.162.96.104
204.162.96.154
204.162.96.176
209.247.194.35
209.247.194.100
64.208.35.5

11:47 pm on Apr 8, 2002 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 3, 2002
posts:105
votes: 0


Wow, Thanks so much, this is what i do:
first i grab the ip address and UA. then i also make the ip a class C. now what I do is check it against a sql2000 database and it does this check.
is IP a translator yes/no
if no
is class C ip like any spiders yes/no
if no
is UA like any spider UAs

This is as thurrow as i can get i think. Anyone else have any better ideas?

11:02 pm on Apr 9, 2002 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Mar 22, 2001
posts:2450
votes: 0


I would do the check against the list of translators before truncating the IP address into a Class C.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members