homepage Welcome to WebmasterWorld Guest from 54.167.138.53
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 169 message thread spans 6 pages: < < 169 ( 1 2 3 4 [5] 6 > >     
Server Farms, Elron Technologies and more
wilderness




msg:4456469
 3:25 pm on May 22, 2012 (gmt 0)

single root request.
No images, no robots.

Visited a few times previously.

207.232.29.zzz - - [22/May/2012:15:27:14 +0100] "GET / HTTP/1.1" 403 559 "-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.14) Gecko/20100824 BonEcho/2.0.0.14"

207.232.0.0 - 207.232.63.255
199.203.0.0 - 199.203.255.255

 

wilderness




msg:4538860
 7:31 am on Jan 24, 2013 (gmt 0)

Versaweb (old thread [webmasterworld.com])

76.164.196.194 - - [Thu Jan 24 06:22:51 2013] "GET / HTTP/1.1" 403 0 "-" "-"

VWEB-208-64-24 208.64.24.0 - 208.64.31.255 208.64.24.0/21
VWEB-208-66-72 208.66.72.0 - 208.66.79.255 208.66.72.0/21
VWEB-72-46-128 72.46.128.0 - 72.46.159.255 72.46.128.0/19
VWEB-76-164-192 76.164.192.0 - 76.164.239.255 76.164.224.0/20 76.164.192.0/19

keyplyr




msg:4538863
 8:06 am on Jan 24, 2013 (gmt 0)

Thanks wilderness

not2easy




msg:4540280
 7:16 pm on Jan 29, 2013 (gmt 0)

I couldn't find anything on INTERBUSINESS less than 10 years old so I thought I would add these here. Please move them if there is somewhere else for this.
I am seeing more activity from INTERBUSINESS, TDENET and RIMA - yes, Rima and TDEnet are telecommunications networks in Spain with legitimate users. The sites they are hitting don't do anything intl so they are blocked:

79.14.0.0 - 79.14.127.255 TELECOM-INTERBUSINESS (IT) 79.14.0.0/16
79.28.128.0 - 79.28.255.255 TELECOM-INTERBUSINESS (IT) 79.28.0.0/16
79.29.128.0 - 79.29.255.255 TELECOM-INTERBUSINESS (IT) 79.29.0.0/16
80.28.128.0 - 80.28.255.255 TDENET (ES) 80.28.0.0/16
80.32.0.0 - 80.35.255.255 RIMA (ES) 80.33.0.0/16
80.36.0.0 - 80.39.255.255 RIMA (ES) 80.36.0.0/16

All of these all well as a few others I'm still checking on are from the past two weeks' access logs for one relatively new WP install and they ALL had the same UA:
"Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1) Gecko/20100101 Firefox/8.0.1"
..and they were all programmed hack attacks: POST /wp-login.php

keyplyr




msg:4540302
 8:15 pm on Jan 29, 2013 (gmt 0)

IMO this forum "Search Engine Spider and User Agent Identification Forum" is for new Search Engines or company/cooperate IP ranges and user agents that may be considered questionable, not for telecom/ISPs that may have a pesky user probing forum or blog security. That's dealt with more on an individual basis.

blend27




msg:4540389
 1:10 am on Jan 30, 2013 (gmt 0)

@keyplyr

I think this thread in particular keeps the "Search Engine" part out of the formula replacing it with "Server Farms, .. and more"..

I have to admit that I personally profited(mentally) from this thing going on and on and on...

I am sure that there are a lot of newbies(or oldies) that lurk in this part of the Woods from time to time started understanding on a more granular level what scraping means to them just based on this thread.

Blend27

wilderness




msg:4540396
 1:56 am on Jan 30, 2013 (gmt 0)

IMO this forum "Search Engine Spider and User Agent Identification Forum" is for new Search Engines or company/cooperate IP ranges and user agents that may be considered questionable, not for telecom/ISPs that may have a pesky user probing forum or blog security. That's dealt with more on an individual basis.


keyplr,
This forum from it's inception (more than a decade ago) (you been here and this should not need any clarification) has been very broad in parameters and has really NEVER followed the forum charters guidelines.

There was even a time when Apache was a primary topic here.

Don

keyplyr




msg:4540427
 4:38 am on Jan 30, 2013 (gmt 0)

As I said, IMO = In My Opinion.

I guess my point is... reporting some guy on an ISP trying to hack into your forum is a personal issue, not reason to post the entire range of the ISP here at WW. I get these guys on a daily basis, but I don't post the ranges of Comcast or Cox Cable.

More specifically, it's doubtful they will still be on that provider a day or two from now.

And I too have benefited from this and other threads, contributing when I can. I don't see what that has to do with what I said. I'm not condemning the thread, just listing ISPs.

Hey, maybe I'm just cranky today... having spent $3k on trademark legal defense a hour ago.

wilderness




msg:4540680
 7:17 pm on Jan 30, 2013 (gmt 0)

NOC4Hosts
blend mentions them in this 2008 thread [webmasterworld.com]

Had the following today:
68.233.255.144 - - [Wed Jan 30 18:13:12 2013] "GET /robots.txt HTTP/1.1" 200 2797 "-" "Mozilla/5.0 (compatible; MSIE 5.01; Windows NT 4.0)"

NOC4HOSTS2 74.50.96.0 - 74.50.127.255 74.50.96.0/19
NOC4HOSTS 199.167.144.0 - 199.167.151.255 199.167.144.0/21
NOC4HOST 198.178.120.0 - 198.178.127.255 198.178.120.0/21
NOC4HOSTS1 68.233.224.0 - 68.233.255.255 68.233.224.0/19
NOC4HOSTS 206.51.224.0 - 206.51.239.255 206.51.224.0/20
NOC4HOSTS1 66.232.96.0 - 66.232.127.255 66.232.96.0/19
NOC4HOST 199.119.100.0 - 199.119.103.255 199.119.100.0/22
NOC4HOSTS1 96.31.64.0 - 96.31.95.255 96.31.64.0/19
NOC4HOST 199.193.112.0 - 199.193.119.255 199.193.112.0/21

dstiles




msg:4540742
 9:41 pm on Jan 30, 2013 (gmt 0)

Thanks - didn't have 3 of those! :)

blend27




msg:4540754
 10:32 pm on Jan 30, 2013 (gmt 0)

@keyplyr
reporting some guy on an ISP trying to hack into your forum is a personal issue


I'll share a personal story here;

Last year I picked up 2 ranges(ISP - comcast/rr) via "to fast scraper block script". Searched the IP in Gorg, one of the theads from WebmasterWorld/this_forum came up. Blocked it on the spot.

Week later found out that one of my customers(dev work) was contacted by an SEO company promising them reaches. SEO company tried to run several custom made stealth tools from Comcast Biz IP Ranges to analyze the site to move on with their kakamimia proposals after I blocked the ranges.

They have approach the client from the angle that I would never pitch. Gorg related, but I wont go into it.

Saved myself a client and got a present for Chanukah.. :)

Just an example.

keyplyr




msg:4540761
 11:10 pm on Jan 30, 2013 (gmt 0)

@blend27 - shall I start listing all ISP ranges from script kiddies, admin.php probes, bogus login attempts ... how about favicon thieves, hot-linking forums?

IMO - This forum is only an asset if it's focused on specific types of threats coming from company ranges that may have negative effects for other webmasters. Reporting some guy on an ISP causing problems on your site is most likely specific to your site alone and not reason to post the entire range of the ISP here at WW.

Nefarious UAs yes, server/colo/data-center ranges yes, new bots yes, because these are the types that affect us all. But as I said above, listing ISPs is not only useless for other webmasters since it is usually a one-time event, but in all probability misleading for those lurking newbies you mention.

@wilderness - thanks for the additional NOC4Hosts ranges. I was missing a couple of those :)

keyplyr




msg:4541037
 6:49 pm on Jan 31, 2013 (gmt 0)



Peer1.net Hosting

65.39.128.0 - 65.39.255.255
65.39.128.0/17

76.74.128.0 - 76.74.255.255
76.74.128.0/17

Any more?

wilderness




msg:4541088
 9:07 pm on Jan 31, 2013 (gmt 0)

Any more?


Peer1 [whois.arin.net]

dstiles




msg:4541121
 10:40 pm on Jan 31, 2013 (gmt 0)

Any More... US, CA and GB...
(note: some ranges include other server providers' sub-ranges)
(the list is almost certainly incomplete)

64.34.0.0-64.34.255.255
64.45.0.0-64.45.63.255
64.65.0.0-64.65.63.255
64.224.0.0-64.227.255.255
65.39.128.0-65.39.255.255
66.33.0.0-66.33.127.255
66.111.64.0-66.111.95.255
66.132.128.0-66.132.255.255
66.199.128.0-66.199.191.255
66.234.0.0-66.234.15.255
67.211.192.0-67.211.207.255
69.0.128.0-69.0.255.255
69.28.192.0-69.28.255.255
69.90.0.0-69.90.255.255
69.172.192.0-69.172.255.255
70.33.192.0-70.33.255.255
72.51.0.0-72.51.63.255
76.74.128.0-76.74.255.255
83.222.224.0-83.222.255.255
107.6.0.0-107.6.63.255
176.74.160.0-176.74.191.255
198.244.48.0-198.244.63.255
209.25.128.0-209.25.255.255
209.203.224.0-209.203.255.255
209.213.96.0-209.213.127.255
216.25.0.0-216.25.127.255
216.65.0.0-216.65.127.255
216.122.0.0-216.122.255.255
216.150.0.0-216.150.31.255
216.152.128.0-216.152.143.255
216.157.0.0-216.157.111.255
216.195.32.0-216.195.63.255

keyplyr




msg:4541170
 2:38 am on Feb 1, 2013 (gmt 0)

Thanks

some ranges include other server providers' sub-ranges

Yes, that's what led me to Peer1. They're a reseller that hosts resellers,
like one of those M.C.Escher drawings :)

blend27




msg:4542014
 2:23 pm on Feb 3, 2013 (gmt 0)

Just caught a crawl attempt from CODERO IPs.

Requests are made from several IPs, 1 second interval.
216.55.161.64
216.55.164.20
206.225.81.153
206.225.81.153
206.225.82.23
216.55.181.182
206.225.94.38
216.55.137.46
216.55.170.28
206.225.93.187
206.225.85.218
206.225.85.162
216.55.181.242
206.225.83.217
216.55.162.116
216.55.161.64
216.55.170.28


With the Same UA: User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:14.0) Gecko/20100101 Firefox/14.0.1

No supporting files, no robots.txt.

Now the interesting part, all IPs when visited redirect to the same shopping site(dudool). The crawler excepts and keeps http Cookie, passes it back on the next visit and provides valid site referrers.

But the bot runner still does not know that his crawler spits out headers that are not humanly possible, all though it really looks like a human visit when looking at the headers.

keyplyr




msg:4542107
 9:14 pm on Feb 3, 2013 (gmt 0)

I also had this one:

Coldero Hosting
216.55.128.0 - 216.55.191.255
216.55.128.0/18

dstiles




msg:4542132
 10:43 pm on Feb 3, 2013 (gmt 0)

Codero:

64.150.176.0 - 64.150.191.255
68.168.96.0 - 68.168.111.255
69.64.64.0 - 69.64.95.255
206.225.80.0 - 206.225.95.255
216.55.128.0 - 216.55.191.255

This list is probably incomplete and I seem to recall a few others ranges either belong to codero or vice versa.

wilderness




msg:4542767
 3:20 am on Feb 6, 2013 (gmt 0)

Steadfast Networks

67.202.90.137 - - [Wed Feb 06 02:41:47 2013] "GET /Myfoleder/MySub/MyPage.html HTTP/1.0" 200 28945 "http://www.example.com/" "Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.02"
67.202.90.137 - - [Wed Feb 06 02:41:53 2013] "GET /SameFolder/SameSub/SamePage.html HTTP/1.0" 200 28945 "http://www.example.com/" "Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.02"

no supporting files. No robots.
Had a visit from one of their other ranges in October for a different page and folder.


STEADFAST-2 208.100.0.0 - 208.100.63.255 208.100.0.0/18
STEADFAST-5 208.117.0.0 - 208.117.63.255 208.117.0.0/18
STEADFAST-FASTROOT 208.66.168.0 - 208.66.175.255 208.66.168.0/21
STEADFAST-1 216.86.144.0 - 216.86.159.255 216.86.144.0/20
STEADFAST-7 23.29.128.0 - 23.29.159.255 23.29.128.0/19
STEADFAST-6 50.31.0.0 - 50.31.127.255 50.31.0.0/17
STEADFAST-3 67.202.64.0 - 67.202.127.255 67.202.64.0/18
STEADFAST-4 69.162.128.0 - 69.162.191.255 69.162.128.0/18
STEADFAST 2607:F128:: - 2607:F128:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

wilderness




msg:4544000
 12:06 am on Feb 9, 2013 (gmt 0)

I'm sure I'm not the only that has seen this.
Checked for WP vulnerabilities four times in a short period, and from four different Class C's.

Pfui mentioned in this thread [webmasterworld.com]

69.163.240.77 - - [Fri Feb 08 21:45:26 2013] "GET /wp-admin/ HTTP/1.1" 403 0 "-" "-"

Dreamhost
DREAMHOST-BLK10 173.236.128.0 - 173.236.255.255 173.236.128.0/17
DREAMHOST-BLK3 205.196.208.0 - 205.196.223.255 205.196.208.0/20
DREAMHOST-BLK5 208.97.128.0 - 208.97.191.255 208.97.128.0/18
DREAMHOST-BLK6 208.113.128.0 - 208.113.255.255 208.113.128.0/17
DREAMHOST-BLK10 64.90.32.0 - 64.90.63.255 64.90.32.0/19
DREAMHOST-BLK4 64.111.96.0 - 64.111.127.255 64.111.96.0/19
DREAMHOST-BLK1 66.33.192.0 - 66.33.223.255 66.33.192.0/19
DREAMHOST-BLK7 67.205.0.0 - 67.205.63.255 67.205.0.0/18
DREAMHOST-BLK9 69.163.128.0 - 69.163.255.255 69.163.128.0/17
DREAMHOST-BLK8 75.119.192.0 - 75.119.223.255 75.119.192.0/19
DREAMHOST-V6-BLK1 2607:F298:: - 2607:F298:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

keyplyr




msg:4544009
 12:41 am on Feb 9, 2013 (gmt 0)



Just a heads-up for those who use the online Wannabrowser.com utility:

Their IP is 69.163.178.111 which is DreamHost (69.163.128.0 - 69.163.255.255)

wilderness




msg:4545829
 5:06 am on Feb 15, 2013 (gmt 0)

Not sure about this one.
I've grouped two of the blocks into larger ranges.

Hostigation
206.253.164.0/22
HOSTIG-1-ARAC1 206.253.164.0 - 206.253.165.255 206.253.164.0/23
HOSTIG-1-ARACH2 206.253.166.0 - 206.253.166.255 206.253.166.0/24
HOSTIG-1-ARACH3 206.253.167.0 - 206.253.167.255 206.253.167.0/24
HSW-216-189-1-0-255 216.189.1.0 - 216.189.1.255 216.189.1.0/24
HSW-216-189-8-0-255 216.189.8.0 - 216.189.8.255 216.189.8.0/24
216-189-101-0-24-HSW 216.189.101.0 - 216.189.101.255 216.189.101.0/24
69.85.64.0/19
HOSTG-GVII4 69.85.84.0 - 69.85.85.255 69.85.84.0/23
HOSTG-GVII3 69.85.86.0 - 69.85.87.255 69.85.86.0/23
HOSTG-GVII2 69.85.88.0 - 69.85.89.255 69.85.88.0/23
HOSTG-GVII26 69.85.91.0 - 69.85.91.255 69.85.91.0/24
HOSTG-GVII1 69.85.92.0 - 69.85.93.255 69.85.92.0/23
HOSTG-GVII25 69.85.95.0 - 69.85.95.255 69.85.95.0/24
HOSTIG-I6-1-ARACH 2606:DF00:2:: - 2606:DF00:2:FFFF:FFFF:FFFF:FFFF:FFFF
HOSTIG-I6-2-ARACH 2606:DF00:3:: - 2606:DF00:3:FFFF:FFFF:FFFF:FFFF:FFFF

keyplyr




msg:4545836
 6:38 am on Feb 15, 2013 (gmt 0)

I think 206.253.164.0 - 206.253.165.255 is 206.253.164.0/23 (not /22)
arachnitec.com is 206.253.160.0 - 206.253.167.255 206.253.160.0/21


and I get Grand Valley Internet (http://gvin.com/) for 69.85.64.0 - 69.85.95.255 69.85.64.0/19 which offers data services for businesses, but the only relation to Hostigation is an arachnitec.com contact email address.

Thoughts?

wilderness




msg:4545837
 6:51 am on Feb 15, 2013 (gmt 0)

keyplr,
look at the hostnames for GrandValley:
EX:
HOSTG-GVII

It appears there is some relationship with host being the backbone.

keyplyr




msg:4545841
 7:26 am on Feb 15, 2013 (gmt 0)

keyplr,
look at the hostnames for GrandValley:
EX:HOSTG-GVII
It appears there is some relationship with host being the backbone.

I don't see it written that way where I checked (I couldn't connect to ARIN) but I'll take your word for it.

Whadoya think about arachnitec.com?

wilderness




msg:4545887
 12:32 pm on Feb 15, 2013 (gmt 0)

keyplr,
It looks to fit this theme as well.

dstiles




msg:4545977
 8:38 pm on Feb 15, 2013 (gmt 0)

216.189.0.0/17 is highspeedweb - blocked here. It includes one or more hostigation sub-ranges.

69.85.64.0/19 is grand valley internet (includes hostigation) - looks server-ish to me but no previous blocking within the range.

keyplyr




msg:4546001
 10:43 pm on Feb 15, 2013 (gmt 0)

Whadoya think about arachnitec.com?
keyplr,
It looks to fit this theme as well.

I meant that it looks more like Hostigation is a sub-range of Arachnitec. Hostigation tech contact evens uses an Arachnitec email address.

Doesn't make much difference I guess.

wilderness




msg:4546013
 11:08 pm on Feb 15, 2013 (gmt 0)

Doesn't make much difference I guess.


A farm is still a farm when the pigs are loose ;)

wilderness




msg:4546256
 3:59 am on Feb 17, 2013 (gmt 0)

There's an old thread on this [webmasterworld.com] by keyplr

Vivid Hosting
VIVID-HOSTING-4 192.154.192.0 - 192.154.255.255 192.154.192.0/18
VIVID-HOSTING-2 192.158.224.0 - 192.158.239.255 192.158.224.0/20
VIVID-HOSTING-3 198.37.96.0 - 198.37.127.255 198.37.96.0/19
VIVID-HOSTING-1 198.177.120.0 - 198.177.127.255 198.177.120.0/21
VIVID-HOSTING 199.188.88.0 - 199.188.95.255 199.188.88.0/21
VIVID-HOSTING-NET 209.133.107.128 - 209.133.107.255 209.133.107.128/25
MZIMA07-CUST-VIVID02 68.64.128.0 - 68.64.129.255 68.64.128.0/23
MZIMA07-CUST-VIVID01 68.64.136.0 - 68.64.137.255 68.64.136.0/23

keyplyr




msg:4546259
 4:34 am on Feb 17, 2013 (gmt 0)



There's an old thread on this [webmasterworld.com] by keyplr

Vivid Hosting
VIVID-HOSTING-4 192.154.192.0 - 192.154.255.255 192.154.192.0/18
VIVID-HOSTING-2 192.158.224.0 - 192.158.239.255 192.158.224.0/20
VIVID-HOSTING-3 198.37.96.0 - 198.37.127.255 198.37.96.0/19
VIVID-HOSTING-1 198.177.120.0 - 198.177.127.255 198.177.120.0/21
VIVID-HOSTING 199.188.88.0 - 199.188.95.255 199.188.88.0/21
VIVID-HOSTING-NET 209.133.107.128 - 209.133.107.255 209.133.107.128/25
MZIMA07-CUST-VIVID02 68.64.128.0 - 68.64.129.255 68.64.128.0/23
MZIMA07-CUST-VIVID01 68.64.136.0 - 68.64.137.255 68.64.136.0/23


So it appears the range I identified in that old thread:

68.64.128.0 - 68.64.191.255
68.64.128.0/18

is now operated by PacketExchange

As is this one:

68.64.128.0 - 68.64.191.255
68.64.128.0/18

And this is AboveNet:

209.133.0.0 - 209.133.127.255
209.133.0.0/17

This 169 message thread spans 6 pages: < < 169 ( 1 2 3 4 [5] 6 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved