homepage Welcome to WebmasterWorld Guest from 54.161.246.212
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
The Idaho crawlers...
I wonder who's behind this
bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4430948 posted 6:34 pm on Mar 19, 2012 (gmt 0)


NetRange: 209.19.128.0 - 209.19.191.255
CIDR: 209.19.128.0/18
OriginAS:
NetName: SPRO-NET-209-19-128
NetHandle: NET-209-19-128-0-1
Parent: NET-209-0-0-0-0
NetType: Direct Allocation
Comment: ADDRESSES WITHIN THIS BLOCK ARE NON-PORTABLE
RegDate: 1997-02-03
Updated: 2012-03-02
Ref: http://whois.arin.net/rest/net/NET-209-19-128-0-1

OrgName: SOLUTION PRO
OrgId: SLPR
Address: 1450 Eagle Flight Way
Address: Ste 200
City: Boise
StateProv: ID
PostalCode: 83709
Country: US
RegDate: 2000-11-22
Updated: 2010-05-18


All with fake reverse dns (which means the owner of the netblock is in on it or delegating without SWIPing... either case is bad)

Had two hits today:

209.19.182.215
209.19.184.215

And I've seen them before from lots of other IP addresses within this block.

UA pre 10-Mar was:

"Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US)"

but has now changed to:

"Mozilla/5.0 (Windows; U; MSIE 7.0; Windows NT 6.0; en-US; 0110120602120527)"

[edited by: bakedjake at 7:39 pm (utc) on Mar 19, 2012]

 

bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4430948 posted 6:43 pm on Mar 19, 2012 (gmt 0)

Another Solution Pro block:

NetRange: 206.80.96.0 - 206.80.127.255
CIDR: 206.80.96.0/19
OriginAS:
NetName: SPRO-NET-206-80-96
NetHandle: NET-206-80-96-0-1
Parent: NET-206-0-0-0-0
NetType: Direct Allocation
RegDate: 1995-08-11
Updated: 2012-03-02
Ref: http://whois.arin.net/rest/net/NET-206-80-96-0-1

OrgName: SOLUTION PRO
OrgId: SLPR
Address: 1450 Eagle Flight Way
Address: Ste 200
City: Boise
StateProv: ID
PostalCode: 83709
Country: US
RegDate: 2000-11-22
Updated: 2010-05-18
Ref: http://whois.arin.net/rest/org/SLPR


Seeing activity from 206.80.118.188. Same UA.

bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4430948 posted 7:11 pm on Mar 19, 2012 (gmt 0)

More crap from Idaho. Same UA. Different IP: 69.5.238.215

NetRange: 69.5.224.0 - 69.5.239.255
CIDR: 69.5.224.0/20
OriginAS:
NetName: ISPEEDWIRELESS
NetHandle: NET-69-5-224-0-1
Parent: NET-69-0-0-0-0
NetType: Direct Allocation
RegDate: 2004-01-20
Updated: 2012-02-24
Ref: http://whois.arin.net/rest/net/NET-69-5-224-0-1

OrgName: ISpeed Wireless Inc.,
OrgId: ISPEE
Address: 112 North Main Street.
City: Payette
StateProv: ID
PostalCode: 83661
Country: US
RegDate: 2003-10-20
Updated: 2009-10-19
Ref: http://whois.arin.net/rest/org/ISPEE

bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4430948 posted 7:13 pm on Mar 19, 2012 (gmt 0)

This crawler is big and distributed, not something one off, so I'm very curious to find out who's behind it.

bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4430948 posted 7:29 pm on Mar 19, 2012 (gmt 0)

More solution pro crap...

NetRange: 207.70.0.0 - 207.70.63.255
CIDR: 207.70.0.0/18
OriginAS:
NetName: SPRO-NET-207-70-0
NetHandle: NET-207-70-0-0-1
Parent: NET-207-0-0-0-0
NetType: Direct Allocation
RegDate: 1996-02-29
Updated: 2012-03-02
Ref: http://whois.arin.net/rest/net/NET-207-70-0-0-1

OrgName: SOLUTION PRO
OrgId: SLPR
Address: 1450 Eagle Flight Way
Address: Ste 200
City: Boise
StateProv: ID
PostalCode: 83709
Country: US
RegDate: 2000-11-22
Updated: 2010-05-18
Ref: http://whois.arin.net/rest/org/SLPR


IP was 207.70.9.102. UA is the same.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4430948 posted 10:23 pm on Mar 19, 2012 (gmt 0)

NetRange: 209.19.128.0 - 209.19.191.255

Funny that you should list this first. I have an ancient-- and probably inaccurate-- block on the whole 209.18.0.0/15 range thanks to something called "reputation.com". Not sure what they did to offend me, but they come around periodically to try the doorknob of some random e-book. Nobody else from the neighborhood has ever tried to visit, so the block evidently has done no harm.

DeeCee



 
Msg#: 4430948 posted 6:02 am on Mar 21, 2012 (gmt 0)

Solution Pro is a CoLocation host, as you mention in Boise. I have policy blocks on ranges of theirs as well. Some very abusive scanners come out of there.

Every time I see Boise, ID passing by on my logger windows, I get suspicious immediately. One of the large Mark Scanner companies own a shell company in that area, which normally runs off its own IP ranges, but running Impersonator bots for the mother ship. Apparently out of a large private home.

It would be very easy for that shell company to stick a server or two in the local coLocation place (Solution Pro) and start running from there as well. Gaining new anonymous IP ranges as a result.

But that is only a suspicion for now. I will have to track the patterns of the bots further to determine whether they really have moved some of their bots out of the house and onto Solution Pro.

But given their behaviors so far, it would be exactly the thing they would do. In their righteous cause of playing detectives searching for bad guys, they do not seem to care how many good sites have to be struck in the head along the way.

bakedjake

WebmasterWorld Administrator bakedjake us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 4430948 posted 12:46 am on Mar 23, 2012 (gmt 0)

More: 206.207.64.0/18

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4430948 posted 1:32 am on Mar 23, 2012 (gmt 0)

More: 206.207.64.0/18


I've had the backbone providers Class B's denied since 2009, however "everybody knows I'm an extremist"

grandma genie



 
Msg#: 4430948 posted 6:49 pm on Aug 17, 2012 (gmt 0)

This is an old post, but thought I would add to it with some more info about this visitor. This guy has been coming onto my site since December, but in June started adding some obfuscated code behind every page request. I'm not going to include that since I don't know what it is doing. But thought I would show some log examples just for your info. Don't have a clue what they are trying to do, but they are always getting served 403s. Just odd that they would come back every day when they never get anything.

First the visitor comes in with head requests and a variety of IPs with just two user agents rotated. Then a second series of Get requests comes in, same series of IPs, but a different Java user agent. Each hit includes a different series of obfuscated code. Here is a sample:

69.5.239.nnn "HEAD /example.html long string of obfuscated code here HTTP/1.1" 403 - "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0)"
207.70.9.nnn "HEAD /example.html long string of obfuscated code here HTTP/1.1" 403 - "-" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; chromeframe/13.0.782.218; chromeframe; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"
207.70.3.nnn
207.70.25.nn
207.70.60.nn
209.19.138.nn
209.19.152.nn
209.19.170.nn
209.19.175.nn
209.19.177.nnn
209.19.178.nnn
209.19.181.nnn
209.19.182.nnn
209.19.184.nnn
209.19.186.nnn
209.19.189.nnn
209.19.191.nn
206.80.112.nnn
206.80.114.nnn
206.80.115.nnn
206.207.116.nn
206.207.117.nnn

69.5.238.nnn "GET /example.html long string of obfuscated code here HTTP/1.1" 403 - "-" "Java/1.6.0_20"
206.80.112.nnn
206.80.118.nnn
206.207.117.nnn
206.207.80.nnn
207.70.9.nnn
207.70.25.nn
207.70.60.nn
209.19.138.nnn
209.19.152.nn
209.19.170.nn
209.19.175.nnn
209.19.180.nnn
209.19.182.nnn
209.19.184.nnn
209.19.186.nnn
209.19.188.nnn
209.19.190.nnn
209.19.178.nnn

They obviously don't come in arranged numerically as I am showing them. I just wanted you to be able to see the variety of IPs. Has anyone seen this type of activity with the objuscated code included?
-- gg

grandma genie



 
Msg#: 4430948 posted 6:50 pm on Aug 17, 2012 (gmt 0)

Oh, by the way, they are always asking for the same three pages. They do not ask for any images. Just html files.

1script

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4430948 posted 7:24 pm on Aug 23, 2012 (gmt 0)

Here is the firewall rule I have for them, did I miss any?

iptables -A INPUT -s 209.19.128.0/18 -j DROP
iptables -A INPUT -s 206.206.0.0/15 -j DROP
iptables -A INPUT -s 207.70.0.0/18 -j DROP
iptables -A INPUT -s 206.80.96.0/18 -j DROP
iptables -A INPUT -s 69.5.224.0/20 -j DROP

service iptables save
service iptables restart


unfortunately, 69.5.x.x comes up as ISpeed Wireless Inc., which at least in theory sounds like it could be a legit ISP, but the same crap comes out of it at the same time as other SOLUTION PRO bots, so I think they're in on it, too. So, they're out as well.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved