homepage Welcome to WebmasterWorld Guest from 23.20.63.27
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Stealth crawler out of Cogent / PSINet 38.99.82.191
carson63000




msg:4650575
 11:41 pm on Mar 2, 2014 (gmt 0)

Hi all,

Wondering if anyone has seen this irritating fellow.

Since I started blocking any "Java/1.?.?_?" User-Agent, I saw a lot of requests blocked from 38.99.82.191 (apparently assigned to PSINet, Inc. by Cogent Communications?)

So I blocked that IP, and saw a rapid string of requests being blocked, with a mix of Java User-Agents (1.6.0_26, 1.6.0_29, 1.6.0_30) and fake Safari User-Agent "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Qt/4.7.1 Safari/533.3". Also a few with just "Mozilla/5.0".

Was wondering if anyone else had come across this? Project Honeypot has a few comments about crawling from there dating back to October 2012.

 

wilderness




msg:4650597
 1:13 am on Mar 3, 2014 (gmt 0)

One of the standard terms to deny access with the UA is Java.

see this thread at top of forum:

Default User Agents of Programming Libraries and Command Line Tools

carson63000




msg:4651214
 3:00 am on Mar 5, 2014 (gmt 0)

Well, indeed, I am denying "Java" UAs, that's how I noticed this IP.

But since blocking the IP, I noticed much, much more traffic was coming from it with a fake browser UA.

I blocked the range 38.99.82.0/23 after seeing it referred to at [networktools.nl...] - the rwhois shows that range as ICEFROST NOC + +1-877-888-9119, ADMIN@ICEFROST.com

icefrost.com is registered to:

System Administrator
IceFrost
9850 S. Maryland Pkwy. A5-358
Las Vegas, NV 89123
UNITED STATES
Telephone: 18778783049
Email:

All the activity I've seen since blocking has been from 38.99.82.191 and 38.99.82.227

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved