homepage Welcome to WebmasterWorld Guest from 54.167.174.90
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Marketing and Biz Dev / Cloaking
Forum Library, Charter, Moderator: open

Cloaking Forum

    
Problem with class C Ip Checks
johnhamman

10+ Year Member



 
Msg#: 337 posted 11:58 pm on Apr 6, 2002 (gmt 0)

Hi All, Question. If i take the ip address of the spider visiting, then knock off the end if it and make it a class c ip. ( 204.152.191.28 to 204.152.191) then do a search in the database it will flag that its a spider because of the folowing ips:204.152.191.41,204.152.191.47, 204.152.191.57, and 204.152.191.58. But Its altavistas translator service!
so whats the best way to get past this?

my pseudocode is:
function SpiderVerify(ip,UA)
'ip = ip address
'ua = User Agent
make ip class C
'123.123.123.123 to 123.123.123
check to see if any ips like that in database.
flag good for no matches.
flag bad for matches.

 

johnhamman

10+ Year Member



 
Msg#: 337 posted 12:18 am on Apr 7, 2002 (gmt 0)

Also why make it a class C instead of the full IP?

Air

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 337 posted 5:41 am on Apr 8, 2002 (gmt 0)

The advantage is that spiders tend to be added from the same class "C" IP range, so it provides an opportunity to detect spiders that would otherwise be missed.

volatilegx

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 337 posted 4:23 pm on Apr 8, 2002 (gmt 0)

You should keep a database of translators as well. Do a check on the translator database before the spider database. Here's the IPs I have listed as translators:

204.123.9.65
204.123.9.66
204.123.9.67
204.123.9.68
204.123.9.106
204.123.9.107
204.152.191.27
204.152.191.28
204.152.191.29
204.152.190.27
204.152.190.28
204.152.190.29
204.152.190.37
204.152.190.154
204.162.96.104
204.162.96.154
204.162.96.176
209.247.194.35
209.247.194.100
64.208.35.5

johnhamman

10+ Year Member



 
Msg#: 337 posted 11:47 pm on Apr 8, 2002 (gmt 0)

Wow, Thanks so much, this is what i do:
first i grab the ip address and UA. then i also make the ip a class C. now what I do is check it against a sql2000 database and it does this check.
is IP a translator yes/no
if no
is class C ip like any spiders yes/no
if no
is UA like any spider UAs

This is as thurrow as i can get i think. Anyone else have any better ideas?

volatilegx

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 337 posted 11:02 pm on Apr 9, 2002 (gmt 0)

I would do the check against the list of translators before truncating the IP address into a Class C.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Marketing and Biz Dev / Cloaking
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved