homepage Welcome to WebmasterWorld Guest from 54.167.174.90
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 327 message thread spans 11 pages: 327 ( [1] 2 3 4 5 6 7 8 9 ... 11 > >     
Server Farms - Sept. 2013
Ongoing Hosting Data Center Discussion
incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:55 pm on Sep 4, 2013 (gmt 0)

Continuation of the May 2013 thread:
[webmasterworld.com...]

 

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 1:20 am on Sep 8, 2013 (gmt 0)

as5577.net, Luxembourg
(hosted-by.buyurl.net)

94.242.192.0 - 94.242.255.255
94.242.192.0/18

More BuyURL

213.238.175.0 - 213.238.175.255
213.238.175.0/24
217.195.202.0 - 217.195.202.127
217.195.202.0/25

Any more?

not2easy

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 4:11 pm on Sep 8, 2013 (gmt 0)

I show the 217.195.202. Buyurl range as:
217.195.202.0/24Buyurl Dedicated Networks217.195.202.0 - 217.195.202.255

route: 217.195.202.0/24
origin: AS20649
source: RIPE # Filtered
When I double checked just now I see it has changed since June and now shows "Fibersunucu Broadband Network Statement" where it had "Buyurl Dedicated Networks" in June.

I did not have the 213.238.175 range and the 94.242. was listed as LU-ROOT, thank you for the update.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 7:02 pm on Sep 8, 2013 (gmt 0)


Thanks not2easy

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:24 pm on Sep 8, 2013 (gmt 0)


Digital Ocean Cloud Servers
37.139.24.0 - 37.139.31.255
37.139.24.0/21

i3d.net Servers/Colo
213.163.71.0 - 213.163.71.127
213.163.71.0/25

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 9:37 pm on Sep 9, 2013 (gmt 0)

I have the range 37.139.0.0 - 37.139.63.255 blocked as petersburginternet RU but (probably) sub-let to DE, LT etc.

And 213.163.64.0 - 213.163.95.255 blocked as interactive3d NL.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 8:09 pm on Sep 13, 2013 (gmt 0)

65.181.64.0/18 Solar VPS

Specific trigger:

65.181.120.179 <snip> :29 <snip> / HTTP/1.1" 200 6338 "-" "AdnormCrawler www.adnorm.com/crawler"
65.181.120.179 <snip> :35 <snip> /images/WorldsHeadline.png HTTP/1.1" 200 1384 <same>
65.181.120.179 <snip> :35 <snip> /images/ducttape.png HTTP/1.1" 200 23856 <same>
65.181.120.179 <snip> :44 <snip> /robots.txt HTTP/1.1" 200 1051 <same>

My logs will sometimes hiccup w/r/t single seconds--shared hosting--but this one's clearly a fifteen-second lag between first request and "Oh, oops, guess I should ask for robots.txt".
keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 8:05 am on Sep 14, 2013 (gmt 0)


I have the range 37.139.0.0 - 37.139.63.255 blocked as petersburginternet RU but (probably) sub-let to DE, LT etc.

dstiles - have you found other server farms within this range, or has it been just independent trouble makers that has led you to block this ISP?

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:19 am on Sep 14, 2013 (gmt 0)

lucy,
had this UA and different Class C on Aug 1.

65.181.124.137 - - [01/Aug/2013:08:06:41 -0600] "GET /MySub/MySubSub/MyPage.html HTTP/1.1" 200 13000 "http://www.example.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10"

More IP's:
SOLAR-VPS 65.181.64.0 - 65.181.127.255 65.181.64.0/18
208-116-29-0-24-FITX Solar VPS (SVL-7) 208.116.29.0 - 208.116.29.255 208.116.29.0/24
SOLAR-VPS 66.187.96.0 - 66.187.111.255 66.187.96.0/20
SOLAR-VPS 209.140.128.0 - 209.140.191.255 209.140.128.0/18
208-116-23-0-24-FITX 208.116.23.0 - 208.116.23.255 208.116.23.0/24
SOLAR-VPS 2605:9800:: - 2605:9800:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF
NET-ID Solar VPS (SVL-7) 69.194.192.0 - 69.194.207.255 69.194.192.0/20

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 10:02 pm on Sep 14, 2013 (gmt 0)

keyplr - as I recall they all seemed, at the time, to belong to the same organisation. Looking again I see that the first half is (now?) Digital Ocean, which includes a cloud range so ipso facto is evil. :)

The second half is petersburgh. If you check DNS for (say) 37.139.48.0 then it returns the full /19 for petersburghinternet despite various /24 ranges saying otherwise.

wilderness - I was missing one of those - thanks. I have the two 208.116 ranges in a single Fortressx range of 208.116.0.0 - 208.116.63.255.

jmccormac

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 4:26 am on Sep 15, 2013 (gmt 0)

This is what FortressITX shows here:
63.140.0.0 - 63.140.15.255
69.194.160.0 - 69.194.175.255
216.67.224.0 - 216.67.255.255
69.57.160.0 - 69.57.191.255
208.116.0.0 - 208.116.63.255
65.98.0.0 - 65.98.127.255
69.72.128.0 - 69.72.255.255
50.60.0.0 - 50.61.255.255

Regards...jmcc

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 6:15 pm on Sep 15, 2013 (gmt 0)

Agreed.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 7:52 pm on Sep 16, 2013 (gmt 0)

SilverServer, Austria

86.59.0.0/17
213.235.192.0/18

Anyone know if this is the extent of these ranges?

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 7:22 pm on Sep 17, 2013 (gmt 0)

SilverServer...

78.142.128.0 - 78.142.191.255
86.59.0.0 - 86.59.127.255
188.118.192.0 - 188.118.255.255
213.235.192.0 - 213.235.255.255

All blocked, although at one time I thought the 188 range may have been DSL.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 7:59 pm on Sep 17, 2013 (gmt 0)



Excellent, thanks dstiles

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 3:58 pm on Sep 18, 2013 (gmt 0)

A new scraper bot on a Rackspace cloud, I blocked on UA, but for which I only have the IP 108.171.187.159

108.171.187.159 - - [16/Sep/2013] "GET /example.htm HTTP/1.1" 403 324 "-" "Mozilla/5.0 (compatible; Embedly/0.2; +http://support.embed.ly/)"

I'd be grateful if anybody could convert that to the full Rackspace CIDR since I've drawn a blank.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 6:28 pm on Sep 18, 2013 (gmt 0)

This is that respective RackSpace range I have blocked:

108.171.160.0 - 108.171.191.255
108.171.160.0/19

BTW - I also have "embed" in UA, referrer, or file path blocked.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 8:59 pm on Sep 18, 2013 (gmt 0)

I've had that range blocked since March last year. I thought I'd posted my rackspace ranges before but probably in an old thread and I think I have some new ones (includes US and UK/European...

5.79.0.0 - 5.79.63.255
31.222.128.0 - 31.222.191.255
37.188.96.0 - 37.188.127.255
46.38.160.0 - 46.38.191.255
50.56.0.0 - 50.57.255.255
64.39.0.0 - 64.39.31.255
64.49.192.0 - 64.49.255.255
65.61.128.0 - 65.61.191.255
66.216.64.0 - 66.216.127.255
67.192.0.0 - 67.192.255.255
67.207.128.0 - 67.207.223.255
69.20.0.0 - 69.20.127.255
72.3.128.0 - 72.3.255.255
72.4.112.0 - 72.4.127.255
72.32.0.0 - 72.32.255.255
74.205.0.0 - 74.205.127.255
78.136.0.0 - 78.136.63.255
89.234.0.0 - 89.234.63.255
92.52.64.0 - 92.52.127.255
94.236.0.0 - 94.236.127.255
95.138.128.0 - 95.138.191.255
98.129.0.0 - 98.129.255.255
108.166.0.0 - 108.166.127.255
108.171.160.0 - 108.171.191.255
120.136.32.0 - 120.136.47.255
162.13.0.0 - 162.13.15.255
162.209.0.0 - 162.209.127.255
164.177.128.0 - 164.177.159.255
166.78.0.0 - 166.78.255.255
173.203.0.0 - 173.203.255.255
174.143.0.0 - 174.143.255.255
180.150.128.0 - 180.150.159.255
184.106.0.0 - 184.106.255.255
192.237.128.0 - 192.237.255.255
198.61.128.0 - 198.61.255.255
198.101.128.0 - 198.101.255.255
204.232.128.0 - 204.232.255.255
207.97.192.0 - 207.97.255.255
209.61.128.0 - 209.61.191.255
209.114.32.0 - 209.114.63.255
212.64.128.0 - 212.64.159.255
212.100.224.0 - 212.100.255.255

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 12:22 am on Sep 19, 2013 (gmt 0)



Thanks dstiles, I didn't have two of those ranges :)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 4:26 am on Sep 19, 2013 (gmt 0)


Voxility

93.114.40.0/21
93.114.40.0 - 93.114.47.255

109.163.233.0/24
09.163.233.0 - 109.163.233.255

109.163.236.0/23
109.163.236.0 - 109.163.237.255

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 4:37 am on Sep 19, 2013 (gmt 0)


scalabledns.com

192.80.128.0/18
192.80.128.0 - 192.80.191.255

owned by Enzu Inc, a cloud hosting provider. Any more ranges?

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 7:21 am on Sep 19, 2013 (gmt 0)



Enzu ranges supplied to me by a fellow WW member:

23.88.0.0/15
23.88.0.0 - 23.89.255.255

172.246.0.0/16
172.246.0.0 - 172.246.255.255

192.80.128.0/18
192.80.128.0 - 192.80.191.255

192.157.192.0/18
192.157.192.0 - 192.157.255.255

198.56.128.0/17
198.56.128.0 - 198.56.255.255

198.71.80.0/20
198.71.80.0 - 198.71.95.255

198.98.96.0/19
198.98.96.0 - 198.98.127.255

199.16.185.0/28
199.16.185.0 - 199.16.185.15

199.48.68.0/22
199.48.68.0 - 199.48.71.255

199.188.72.0/22
199.188.72.0 - 199.188.75.255

199.193.248.0/21
199.193.248.0 - 199.193.255.255

199.229.232.0/22
199.229.232.0 - 199.229.235.255

199.231.208.0/21
199.231.208.0 - 199.231.215.255

Any more?

Angonasec

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4607411 posted 8:49 am on Sep 19, 2013 (gmt 0)

Thanks for the Rackspace CIDR updates. I did trawl the old mssgs b4 posting.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 9:12 am on Sep 19, 2013 (gmt 0)

I thought I'd posted my rackspace ranges before

Turns out I've got some of those ranges blocked from years ago, with the label Slicehost. Are they the same people? For, ahem, a given definition of "people".

jmccormac

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 9:34 am on Sep 19, 2013 (gmt 0)

Some of Rackspace's ranges have Slicehost netnames/nethandles from what I remember. I think that Rackspace took over a few companies in the past but it has a lot of IP addresses and some companies use its IP space.

Regards...jmcc

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 7:16 pm on Sep 19, 2013 (gmt 0)

Voxility...

37.43.44.0 - 37.43.44.255
37.221.160.0 - 37.221.175.255
93.114.40.0 - 93.114.47.255
103.31.184.0 - 103.31.187.255
109.163.224.0 - 109.163.239.255
185.5.172.0 - 185.5.175.255
195.60.76.0 - 195.60.77.255

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 8:30 pm on Sep 19, 2013 (gmt 0)

Thanks dstiles

not2easy

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 4:20 am on Sep 20, 2013 (gmt 0)

I also have 93.115.92.0 - 93.115.95.255 for Voxility since July. Thanks for the list, I was missing two of them.

Sorry about the double post, guess I took too long editing.

[edited by: not2easy at 4:42 am (utc) on Sep 20, 2013]

not2easy

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 4:39 am on Sep 20, 2013 (gmt 0)

I also have 93.115.92.0 - 93.115.95.255 for Voxility since July. Thanks for the list, I was missing two of them.

Another - I missed this back on the 14th, but I see an older note on hosting in the 37.139 space:
GIGA-HOSTING MRHOSTBIZ-NET DE37.139.52.0 - 37.139.52.255scraper - - [02/Jan/2013

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 5:49 am on Sep 20, 2013 (gmt 0)

@not2easy - Thanks for the additional Voxility range. The MrHost.biz/GigaHosting I have Contabo as the *new* big cheese:

37.139.52.0/24
37.139.52.0 - 37.139.52.255

(contabo.com and contabo.de)


Another Contabo (formerly GigaHosting as well) range is:

91.194.90.0/22
91.194.90.0 - 91.194.91.255

This 327 message thread spans 11 pages: 327 ( [1] 2 3 4 5 6 7 8 9 ... 11 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved