homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

Stealth crawler out of Cogent / PSINet

Msg#: 4650573 posted 11:41 pm on Mar 2, 2014 (gmt 0)

Hi all,

Wondering if anyone has seen this irritating fellow.

Since I started blocking any "Java/1.?.?_?" User-Agent, I saw a lot of requests blocked from (apparently assigned to PSINet, Inc. by Cogent Communications?)

So I blocked that IP, and saw a rapid string of requests being blocked, with a mix of Java User-Agents (1.6.0_26, 1.6.0_29, 1.6.0_30) and fake Safari User-Agent "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Qt/4.7.1 Safari/533.3". Also a few with just "Mozilla/5.0".

Was wondering if anyone else had come across this? Project Honeypot has a few comments about crawling from there dating back to October 2012.



WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 4650573 posted 1:13 am on Mar 3, 2014 (gmt 0)

One of the standard terms to deny access with the UA is Java.

see this thread at top of forum:

Default User Agents of Programming Libraries and Command Line Tools


Msg#: 4650573 posted 3:00 am on Mar 5, 2014 (gmt 0)

Well, indeed, I am denying "Java" UAs, that's how I noticed this IP.

But since blocking the IP, I noticed much, much more traffic was coming from it with a fake browser UA.

I blocked the range after seeing it referred to at [networktools.nl...] - the rwhois shows that range as ICEFROST NOC + +1-877-888-9119, ADMIN@ICEFROST.com

icefrost.com is registered to:

System Administrator
9850 S. Maryland Pkwy. A5-358
Las Vegas, NV 89123
Telephone: 18778783049

All the activity I've seen since blocking has been from and

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved