homepage Welcome to WebmasterWorld Guest from 50.17.7.84
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 327 message thread spans 11 pages: < < 327 ( 1 2 3 4 5 6 7 8 9 [10] 11 > >     
Server Farms - Sept. 2013
Ongoing Hosting Data Center Discussion
incrediBILL




msg:4607413
 11:55 pm on Sep 4, 2013 (gmt 0)

Continuation of the May 2013 thread:
[webmasterworld.com...]

 

keyplyr




msg:4637808
 9:06 am on Jan 16, 2014 (gmt 0)

Looks to me like a 3rd party CMS specifically to add content to a web page. Presumably audio & video media, articles & content, etc. What lead me to blocking them (aside from the AWS address which I do poke a few holes in for the occasional "exception") is this statement on the home page:

Embed turns any URL into embeddable content


IMO That implies they scrape snippets (or more) from outgoing links.

blend27




msg:4639766
 3:28 am on Jan 24, 2014 (gmt 0)

WORLDSTREAM from NL

93.190.136.0 - 93.190.143.255 NL-WORLDSTREAM-20080516 93.190.136.0/22
217.23.0.0 - 217.23.15.255 NL-WORLDSTREAM-20090204 217.23.0.0/20
109.236.80.0 - 109.236.95.255 NL-WORLDSTREAM-20091204 109.236.80.0/20

dstiles




msg:4640002
 9:48 pm on Jan 24, 2014 (gmt 0)

I have an extra one, making...

84.243.192.0 - 84.243.255.255
93.190.136.0 - 93.190.143.255
109.236.80.0 - 109.236.95.255
217.23.0.0 - 217.23.15.255

not2easy




msg:4640046
 6:04 am on Jan 25, 2014 (gmt 0)

A new Digital Ocean range:
107.170.0.0 - 107.170.255.255
107.170.0.0/16
shows a reg date 12/30/13

bobothecat2




msg:4640105
 4:08 pm on Jan 25, 2014 (gmt 0)

I have an extra one, making...

84.243.192.0 - 84.243.255.255
93.190.136.0 - 93.190.143.255
109.236.80.0 - 109.236.95.255
217.23.0.0 - 217.23.15.255


I too have a few extras, making...

91.226.30.0 - 91.226.31.255
93.170.13.0 - 93.170.13.255
93.170.77.0 - 93.170.77.255
93.190.136.0 - 93.190.143.255
109.236.80.0 - 109.236.95.255
192.71.151.0 - 192.71.151.255
217.23.0.0 - 217.23.15.255

dstiles




msg:4640147
 7:52 pm on Jan 25, 2014 (gmt 0)

93.170.0.0/15 is alfatelecom partly assigned to worldstream. See eariler entry.

not2easy




msg:4640675
 5:50 am on Jan 28, 2014 (gmt 0)

Caught something hosted at this:
FortaTrust USA (Doral, FL)
198.154.60.0 - 198.154.63.255198.154.60.0/22
trying to POST a sql injection.
A quick check says they offer dedicated, cloud and colo hosting - and are bringing more servers online across the globe. great.

Angonasec




msg:4640709
 10:14 am on Jan 28, 2014 (gmt 0)

50.2.223.nn - - [28/Jan/2014] "GET /example.htm HTTP/1.0" 200 6101 "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"

ServerHub Cloud OVZ Dallas 50.2.220.0 - 50.2.229.255
Eonix Corporation 50.2.0.0 - 50.3.255.255 50.2.0.0/15

That's enough to get Blocked from now on.

dstiles




msg:4640857
 10:09 pm on Jan 28, 2014 (gmt 0)

Interestingly, 198.154.60.0 - 198.154.63.255 is in the middle of a DoD block. Any nefarious connection, I wonder?

lucy24




msg:4640880
 12:13 am on Jan 29, 2014 (gmt 0)

Just met someone from "root SA" in, of all places, Luxembourg:

212.117.160.0/19

Are there more of them? I think we can stipulate that "root" is not a useful search term, though I did try.

not2easy




msg:4640899
 1:15 am on Jan 29, 2014 (gmt 0)

@lucy24 I see that as eSolutions, it was blocked for taking things it was blocked from on one site. I don't find anything else in my records for eSolutions, but that is the same CIDR.

not2easy




msg:4640902
 1:26 am on Jan 29, 2014 (gmt 0)

re: DoD, I never heard of FortaTrust so I asked Ixquick and got a page full of hosting solutions links, all referencing the Doral loc. I didn't dig any deeper, just wanted to be sure it wasn't a telecom so I'd know how to block them.

lucy24




msg:4640922
 5:40 am on Jan 29, 2014 (gmt 0)

95.143.192.0/20
ServerConnect (Sweden)

Is there really just one of it? Forums search comes up cold; I even cross-checked by looking for the IP.


Elsewhere:
109.237.128.0/20
AlfaHosting (Germany)
... et cetera, as above ;) Free lookup says "1,892 websites use this address". And at least one robot.

wilderness




msg:4641007
 2:05 pm on Jan 29, 2014 (gmt 0)

re: DoD, I never heard of FortaTrust


FortaTrust USA Corporation
FT-USA-DR2 198.154.60.0 - 198.154.63.255 198.154.60.0/22
FUC-US-2001 162.213.152.0 - 162.213.155.255 162.213.152.0/22
FT-IP6-1002 2607:9300:: - 2607:9300:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF
FUC-US-1001 199.195.212.0 - 199.195.215.255 199.195.212.0/22

wilderness




msg:4641011
 2:30 pm on Jan 29, 2014 (gmt 0)

ServerHub Cloud OVZ Dallas 50.2.220.0 - 50.2.229.255
Eonix Corporation 50.2.0.0 - 50.3.255.255 50.2.0.0/15


Eonix is the backbone:

EONIX-NET-107-158-0-0-1-BLK-10 107.158.0.0 - 107.158.255.255 107.158.0.0/16
EONIX-NET-173-44-128-0-1-BLK-4 173.44.128.0 - 173.44.255.255 173.44.128.0/17
EONIX-NET-173-213-64-0-1-BLK-3 173.213.64.0 - 173.213.127.255 173.213.64.0/18
EONIX-NET-173-232-0-0-1-BLK-6 173.232.0.0 - 173.232.255.255 173.232.0.0/16
EONIX-NET-206-214-64-0-1-BLK-2 206.214.64.0 - 206.214.95.255 206.214.64.0/19
EONIX-NET-208-89-216-0-1-BLK-1 208.89.216.0 - 208.89.223.255 208.89.216.0/21
EONIX-NET-23-231-0-0-1-BLK-9 23.231.0.0 - 23.231.127.255 23.231.0.0/17
EONIX-NET-23-90-0-0-1-BLK-8 23.90.0.0 - 23.90.63.255 23.90.0.0/18
EONIX-NET-50-2-0-0-1-BLK-7 50.2.0.0 - 50.3.255.255 50.2.0.0/15
EONIX-NET-75-75-224-0-1-BLK-5 75.75.224.0 - 75.75.255.255 75.75.224.0/19
EONIX-NET-2607-FF28-BLK-V6-1 2607:FF28:: - 2607:FF28:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

wilderness




msg:4641012
 2:40 pm on Jan 29, 2014 (gmt 0)

Are there more of them? I think we can stipulate that "root" is not a useful search term, though I did try.


lucy,
ARIN use to allow "sub net searches", however no longer does.

I've never been able to determine "sub net search" capability at either ARIN or RIPE.

Here's an OLD example of a sub-net search at ARIN:

> 63.144.

which provided everything below and including 63.144.

Angonasec




msg:4641023
 3:14 pm on Jan 29, 2014 (gmt 0)

Thank you for the further Eonix data, Mr. Wilderness:

Another day, another sneaky West Coast cowboy masquerading as a comm company.

This one has the catchy name: xeex (They just couldn't resist that domain when they found it was not taken.)

Identical activity to Eonix Corp posted above.

Like the Eonix bot, it hit just the +one file+, no css, no js, no images.

Identical UA.

216.152.254.nn - - [28/Jan/2014] "GET /example.htm HTTP/1.1" 200 6101 "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)"

xeex 216.152.240.0 - 216.152.255.255 216.152.240.0/20 Blocked

wilderness




msg:4641025
 3:25 pm on Jan 29, 2014 (gmt 0)

xeex 216.152.240.0 -



XEEX-COMMUNICATIONS 162.213.128.0 - 162.213.131.255 162.213.128.0/22
TRITN-208-75-90-0-24 208.75.90.0 - 208.75.90.255 208.75.90.0/24
TRITN-209-159-130-0-23 209.159.130.0 - 209.159.131.255 209.159.130.0/23
TRITN-209-159-133-0-24 209.159.133.0 - 209.159.133.255 209.159.133.0/24
TRITN-209-159-140-0-23 209.159.140.0 - 209.159.141.255 209.159.140.0/23
XEEX-COMMUNICATIONS 69.26.160.0 - 69.26.191.255 69.26.160.0/19
XEEX-COMMUNICATIONS 69.26.172.0 - 69.26.175.255 69.26.172.0/22
XEEX-COMMUNICATIONS 216.151.128.0 - 216.151.159.255 216.151.128.0/19
XEEX-COMMUNICATIONS 216.152.240.0 - 216.152.255.255 216.152.240.0/20
XEEX-6NETBLK 2607:F2D0:: - 2607:F2D0:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Angonasec




msg:4641028
 3:40 pm on Jan 29, 2014 (gmt 0)

You won't be surprised to hear of yet another bot performing the identical routine to the Eonix and the xeex bots, this one is run by those excitable Swedish chaps WEBEXXPURTS who have found a US server on the Pacific Network.


130.185.158.nnn - - [29/Jan/2014] "GET /example.htm HTTP/1.0" 200 6101 "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 3.5.30729)"

Slightly different UA but same sneaky activity.

Pacific Network 130.185.156.0 - 130.185.159.255 130.185.156.0/22 Blocked

thetrasher




msg:4641081
 8:53 pm on Jan 29, 2014 (gmt 0)

Chinese servers in USA
Federal Online Group LLC

23.231.128.0 - 23.231.255.255 = 23.231.128.0/17
107.163.0.0 - 107.163.255.255 = 107.163.0.0/16
192.155.160.0 - 192.155.191.255 = 192.155.160.0/19
192.186.0.0 - 192.186.63.255 = 192.186.0.0/18
192.250.192.0 - 192.250.207.255 = 192.250.192.0/20
192.250.240.0 - 192.250.255.255 = 192.250.240.0/20

As Nobis Technology Group customer
23.19.43.64 - 23.19.43.71 = 23.19.43.64/29
23.81.146.0 - 23.81.146.255 = 23.81.146.0/24
108.62.112.88 - 108.62.112.95 = 108.62.112.88/29
108.62.170.128 - 108.62.170.143 = 108.62.170.128/28
142.234.138.0 - 142.234.138.255 = 142.234.138.0/24

lucy24




msg:4641084
 9:00 pm on Jan 29, 2014 (gmt 0)

Oh, good. I like MSIE 6. It means that even if I've never heard of them before, all they ever get is the old-browser page.

not2easy




msg:4641321
 8:35 pm on Jan 30, 2014 (gmt 0)

Thank you for the FortaTrust I didn't have, wilderness! Here's a new Codero I didn't have. It's not a new reg so many may have it, but I had nothing on this one:
Codero
66.226.72.0 - 66.226.79.25566.226.72.0/21

dstiles




msg:4641331
 9:04 pm on Jan 30, 2014 (gmt 0)

wilderness - try robtex. It used to allow cnet (nnn.nnn.nnn) and I think nnn.nnn as well but I haven't used it for a while so do not know if it still does.

xeex - I also have an India one at 113.212.64.0 - 113.212.95.255

WEBEXXPURTS:

5.34.240.0 - 5.34.247.255
5.153.232.0 - 5.153.239.255
5.157.0.0 - 5.157.63.255
37.72.184.0 - 37.72.191.255
37.203.208.0 - 37.203.215.255
46.29.248.0 - 46.29.255.255
130.185.156.0 - 130.185.159.255
151.237.176.0 - 151.237.191.255
176.61.136.0 - 176.61.143.255
178.216.48.0 - 178.216.55.255
185.3.132.0 - 185.3.135.255

I have the codero range 66.226.72.0 - 66.226.79.255 as part of the InternetNames range 66.226.64.0 - 66.226.95.255.

wilderness




msg:4641408
 2:51 am on Jan 31, 2014 (gmt 0)

try robtex. It used to allow cnet (nnn.nnn.nnn) and I think nnn.nnn as well but I haven't used it for a while so do not know if it still does.


dstiles,
Don't quite understand what your asking and/or explaining here?

I've a solitary reference to robtex from 2004 and that came from a Beyond The Network America IP.

lucy24




msg:4641419
 4:16 am on Jan 31, 2014 (gmt 0)

robtex dot com. Assorted lookups. Feed in some values at random and you'll see... well, something. I don't perfectly understand why they say 74.xyz is AfriNIC even while they spit out a string of Verizon IPs.

Unless, that is, there's also a robtex dot some-other-tld that I didn't try.

wilderness




msg:4641435
 6:04 am on Jan 31, 2014 (gmt 0)

try robtex. It used to allow cnet (nnn.nnn.nnn) and I think nnn.nnn as well but I haven't used it for a while so do not know if it still does.


robtex dot com. Assorted lookups.


Thanks.

Now I see that he's referring to sub-net searches

This is a very lame tool compared to what ARIN used to have.

Angonasec




msg:4641517
 11:54 am on Jan 31, 2014 (gmt 0)

Thank you dtiles:
113. is sino-blocked already

I'll keep those other ranges near to hand.

Angonasec




msg:4641527
 12:48 pm on Jan 31, 2014 (gmt 0)

Identical trauma to Eonix, xeex, and those modest Swedish chaps, today we present EDIS. With a new UA.

Not content with starting only two world-wars these Austrians are busy developing their Icelandic Reich:

EDIS GmbH 151.236.24.0 - 151.236.24.255 151.236.24.0/24 Blocked

151.236.24.nnn - - [31/Jan/2014:07:37:28 -0500] "GET /example.htm HTTP/1.0" 200 6010 "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"

not2easy




msg:4641574
 4:25 pm on Jan 31, 2014 (gmt 0)

@dstiles I have MEGA-11 for 66.226.64.0 - 66.226.71.255 and MEGA-12 for 66.226.80.0 - 66.226.95.255
both are InternetNamesForBusiness so it looks like Codero is tucked in between those two. I ran new lookups because I lacked a date for the MEGA info.

dstiles




msg:4641615
 8:01 pm on Jan 31, 2014 (gmt 0)

Angonasec - HTTP/1.0 is about 90% bot and 10% proxy - filter out the proxies you are willing to accept and the rest can be blocked.

I have four edis ranges...

37.235.48.0 - 37.235.63.0
149.154.152.0 - 149.154.159.255
151.236.0.0 - 151.236.31.255
158.255.208.0 - 158.255.215.255

They seem to host across several EU countries.

not2easy - InternetNamesForBusiness seems to be the host for the full range, covering mega and codero in this case.

Angonasec




msg:4641658
 2:01 am on Feb 1, 2014 (gmt 0)

Ta for the tip, Sir: Done!

This 327 message thread spans 11 pages: < < 327 ( 1 2 3 4 5 6 7 8 9 [10] 11 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved