homepage Welcome to WebmasterWorld Guest from 54.211.97.242
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 327 message thread spans 11 pages: < < 327 ( 1 2 3 4 5 6 7 [8] 9 10 11 > >     
Server Farms - Sept. 2013
Ongoing Hosting Data Center Discussion
incrediBILL

WebmasterWorld Administrator incredibill us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:55 pm on Sep 4, 2013 (gmt 0)

Continuation of the May 2013 thread:
[webmasterworld.com...]

 

Angonasec

10+ Year Member



 
Msg#: 4607411 posted 11:52 am on Dec 19, 2013 (gmt 0)

dstiles: Over 60% of web traffic in 2013 was bots, and a large proportion of that 60% was miscreant. As an avid log-watcher I'm sure you've noticed :)

Only 40% is human traffic, and not all the 40% is virtuous.

The see-saw has tipped.

The gyro toppled.

Nerd-dom and geek-dom beckon.

So those of us who do not wish to become either, eagerly await a better filtering solution than any of us has so far employed.

Meanwhile, keep bailing folks!

A Merry Christmas to all our readers.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 4:08 pm on Dec 19, 2013 (gmt 0)

New Digital Ocean range

95.85.0.0 - 95.85.63.255
95.85.0.0/18
Netherlands (NL)

This was brought to light by the following UA (my [])...

elefent/Elefent 1.2 (A friendly web elefent.; [elefent[.]eu...] webmaster@elefent[.]eu)

This UA was mentioned in a previous thread (now closed) as coming from an ADSL line. Obviously it has moved on.

My total Digital Ocean list is now as follows, split roughly half US and half NL...

37.139.0.0 - 37.139.31.255
82.196.0.0 - 82.196.15.255
95.85.0.0 - 95.85.63.255
146.185.128.0 - 146.185.191.255
162.243.0.0 - 162.243.255.255
185.14.184.0 - 185.14.187.255
192.34.56.0 - 192.34.63.255
192.81.208.0 - 192.81.223.255
192.241.128.0 - 192.241.255.255
198.199.64.0 - 198.199.127.255
198.211.96.0 - 198.211.127.255
208.68.36.0 - 208.68.39.255

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 3:21 am on Dec 22, 2013 (gmt 0)

brand spankin new addition to the WeHost [webmasterworld.com] family. I always like companies that include "Host" in their name. Makes it easier.

23.239.192.0/19

Only registered last month, looks like, and already snuffling around a site I just set up the day before yesterday (really).

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 4:01 am on Dec 22, 2013 (gmt 0)



Lucy I've had that one since late 2003. Possibly the registration info was just updated.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 4:32 am on Dec 22, 2013 (gmt 0)

Are you sure about the numbers? Most of 23. was bogons until recently. There are still huge gaps.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 5:26 am on Dec 22, 2013 (gmt 0)


I'm not sure about anything :)

Only that my notes say...
WeHostWebSites.com 10/20/2003
23.239.192.0/19
23.239.192.0 - 23.239.223.255

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 6:26 am on Dec 22, 2013 (gmt 0)

I've these three new ranges updating my May 18, 2013 reference:

NET-WEHOST-6 162.213.216.0 - 162.213.223.255 162.213.216.0/21
NET-WEHOST-7 192.199.240.0 - 192.199.255.255 192.199.240.0/20
NET-WEHOST-8 23.239.192.0 - 23.239.223.255 23.239.192.0/19

Is it possible that the 2003 range was a subnet of them-larger-backbone?

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 4:03 pm on Dec 22, 2013 (gmt 0)

New Hetzner range: 41.203.0.0 - 41.203.31.255 (ZA)

WeHost - I have the following...

23.239.192.0 - 23.239.223.255
64.27.48.0 - 64.27.63.255
68.71.128.0 - 68.71.159.255
72.18.128.0 - 72.18.159.255
162.213.216.0 - 162.213.223.255
173.248.128.0 - 173.248.191.255
192.82.248.0 - 192.82.255.255
192.199.240.0 - 192.199.255.255

Three of those APPEAR recent, in that I didn't have them listed (ie they hadn't attacked me) until November/December. Perhaps they are just better managed than many. :)

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 7:47 am on Dec 24, 2013 (gmt 0)


Another annon service

anchorfree.com
74.115.0.0/21
74.115.0.0 - 74.115.7.255

199.255.208.0/21
199.255.208.0 - 199.255.215.255

Surely they have other ranges?

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 6:09 pm on Dec 24, 2013 (gmt 0)

Serverel

There's a 2012 thread by gg [webmasterworld.com], however it's for a APNIC Range.

The following are ARIN ranges:
SERVEREL 162.221.224.0 - 162.221.227.255 162.221.224.0/22
SERVEREL 173.214.240.0 - 173.214.255.255 173.214.240.0/20
SERVEREL 192.133.136.0 - 192.133.143.255 192.133.136.0/21
SERVEREL 199.182.160.0 - 199.182.167.255 199.182.160.0/21
SERVEREL 216.172.48.0 - 216.172.63.255 216.172.48.0/20
SERVEREL-IPV6-NET 2605:5B00:: - 2605:5B00:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 9:07 pm on Dec 24, 2013 (gmt 0)

I don't know if they are all the same company but I have the following listed under serverel, all blocked. They are a mix of US and several european countries.

31.148.128.0 - 31.148.255.255
62.122.168.0 - 62.122.175.0
62.182.160.0 - 62.182.167.255
79.110.64.0 - 79.110.79.255
79.110.96.0 - 79.110.111.255
79.110.208.0 - 79.110.223.255
91.227.144.0 - 91.227.147.255
91.229.94.0 - 91.229.94.255
91.230.216.0 - 91.230.217.255
94.231.192.0 - 94.231.207.255
109.206.160.0 - 109.206.191.255
162.221.224.0 - 162.221.227.255
173.214.240.0 - 173.214.255.255
192.133.136.0 - 192.133.143.255
199.182.160.0 - 199.182.167.255
213.109.192.0 - 213.109.207.255
216.172.48.0 - 216.172.63.255

bobothecat2



 
Msg#: 4607411 posted 9:34 pm on Dec 25, 2013 (gmt 0)

Reprise Hosting

162.248.4.0 - 162.248.7.255 162.248.4.0/22

Anyone have more?

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:59 pm on Dec 25, 2013 (gmt 0)



HP Cloud Services
15.0.0.0/8
15.0.0.0 - 15.255.255.255

15.185.100.174 - - [25/Dec/2013:11:28:08 -0800] "GET /example.html HTTP/1.1" 200 16524 "-" "Mozilla/5.0 (Windows NT 6.0; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0"

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4607411 posted 3:16 pm on Dec 27, 2013 (gmt 0)

New hetzner range:

148.251.0.0 - 148.251.255.255

Angonasec

10+ Year Member



 
Msg#: 4607411 posted 12:23 am on Dec 30, 2013 (gmt 0)

And, lunging for the inside line of the year's final corner, we present a new toxic China-USA range for your delight:
192.151.192.0 - 192.151.255.255 << 192.151.192.0/18

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 1:51 am on Dec 30, 2013 (gmt 0)

New Hetzner range

Isn't there a list somewhere of Top Ten Things Webmasters Do Not Want To Hear, Ever?

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 1:57 am on Dec 30, 2013 (gmt 0)

These threads (due to the inability to link to individual submissions) have become literally worthless, at least in the reference and/or search options.

I did not check, however these CloudRadium "may" be subnets of larger Nobis Tech ranges.

CloudRadium
CLOUDRADIUM-LA4 162.209.128.0 - 162.209.255.255 162.209.128.0/17
CLOUDRADIUM-LA3 172.247.0.0 - 172.247.255.255 172.247.0.0/16
CLOUDRADIUM-CAGE2 192.151.192.0 - 192.151.255.255 192.151.192.0/18
CLOUDRADIUM-USA 192.161.80.0 - 192.161.95.255 192.161.80.0/20
CLOUDRADIUM-LA 198.16.32.0 - 198.16.63.255 198.16.32.0/19
CLOUDRADIUM-5 23.224.0.0 - 23.225.255.255 23.224.0.0/15
CLOUDRADIUM-LA 2604:3880:: - 2604:3880:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF

Angonasec

10+ Year Member



 
Msg#: 4607411 posted 7:48 am on Dec 30, 2013 (gmt 0)

Thanks Mr. Wilderness, not worthless for those of us who remember Mme. Curie's error.

China seems to be buying up struggling US data centres.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 8:04 am on Dec 30, 2013 (gmt 0)

"China seems to be buying up struggling US data centres."

The word "cloud" is changing the interpretation of the internet and/or networks.

Angonasec

10+ Year Member



 
Msg#: 4607411 posted 8:25 am on Dec 30, 2013 (gmt 0)

Indeed; cloud = block me quick or die!

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 8:36 am on Dec 30, 2013 (gmt 0)

I was going to say something about CloudRadium earlier, but got in a fight with my keyboard :(

Not long ago I met an apparent human from that 192.151.192.0/18 range. I got the impression it's a proxy. Cell phone, hot on the heels of 112.79.43.abc (same UA requesting same page) which claims to be India.

due to the inability to link to individual submissions

Post it in the Forums Wishlist [webmasterworld.com] thread ;)

Each post does have a #anchor so there's no inherent reason posts can't be linked; it just requires a code tweak.

For comparison purposes: Earlier this year someone explained to me how a php/bb URL works. Links to a specific post end in a seemingly redundant
?p=123456#123456
but there's a reason for the redundancy. The query part means "the page that contains post 123456" (php/bb forums also have a thread number, but it isn't used in this type of URL), while the fragment means the post, wherever on the page it may be.

Meanwhile at WebmasterWorld, the post directly above mine is
www.webmasterworld.com/search_engine_spiders/4607411-8-30.htm#4634071
This can only be used for linking if you have the same posts-per-page setting that I do. You can change the "-30" part of the URL or even omit it entirely; it won't override your own prefs. I experimented. If your setting is for a higher number, the URL will take you to your page 8, which may not be the same as mine. Or to the last page of the thread, if there aren't 8 pages. So post #4634071 may or may not be present on this page. But it does have an <a name>. (Only a name. No id yet.)

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 9:13 am on Dec 30, 2013 (gmt 0)

lucy,
the individual submission links worked for more than a decade.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 4:46 pm on Dec 30, 2013 (gmt 0)

You mean the part where Forums software eats any fragment links? It doesn't really matter whether it did or didn't work in the past; either way it can be (re)made to work in the future.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 5:04 pm on Dec 30, 2013 (gmt 0)

for the sake of clarity, in html they're called page bookmarks.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 10:41 pm on Dec 31, 2013 (gmt 0)

Today's webexxpurts range
:: yawn ::
5.153.232.0/21

Free lookup gives smaller pieces. But if 5.153.237 is one piece and 5.153.232.0/22 is another, and I already know the /21 is allocated to Sweden, I'm not going to bother looking up 236, 238 and 239.

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 11:05 pm on Dec 31, 2013 (gmt 0)



I've had 5.153.232.0/22 blocked for a month or two... just say'n.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 8:17 am on Jan 1, 2014 (gmt 0)

This a bit off-topic for this thread, however since the activity came from a farm, it fits.

Fortunately between the blank UA and the farm IP, all it did was eat 403's.

Checking for PHP vulnerabilities.
627 successive requests over 35 minutes.
I've not seen anything this persistent in some while.

Choopa, LLC CHOOPA-NETBLK06 209.222.0.0 - 209.222.31.255
ReliableSite.Net LLC NET-209-222-3-168-29 209.222.3.168 - 209.222.3.175

first request (space inserted to break URL)
209.222.3.170 - - [31/Dec/2013:17:39:08 -0700] "GET /0_admin/modules/Wochenkarte/frontend/index.php?x_admindir=http://www. google.com/humans.txt? HTTP/1.0" 403 794 "-" "-"

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 9:32 am on Jan 1, 2014 (gmt 0)

That reminds me that the first time I saw a request for humans.txt, I thought the visitor was being humorous. I've since learned that "humans.txt" is a real thing. (That's an actual google page in there. Was it really part of the request, or did you change it for safety?)

:: wandering off to investigate because I can't imagine why I don't have that Choopa range flagged ::

Ah. Only visitor I've had from there in measurable time was something calling itself the Ahrefsbot-- whether truthfully or otherwise-- so it was locked out a priori and I never even saw it.

Edit: I found this list in another thread.

64.237.32.0 - 64.237.63.255
66.55.128.0 - 66.55.159.255
68.232.160.0 - 68.232.191.255
108.61.0.0 - 108.61.255.255
173.199.64.0 - 173.199.127.255
208.167.224.0 - 208.167.255.255
209.222.0.0 - 209.222.31.255
216.155.128.0 - 216.155.159.255

They must not be uniformly loathsome; some of those numbers I've never set eyes on.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4607411 posted 9:51 am on Jan 1, 2014 (gmt 0)

Was it really part of the request, or did you change it for safety?


lucy,
That exact trailing google URL was in each one of the 627 requests, and was immediately following the PHP request.

Angonasec

10+ Year Member



 
Msg#: 4607411 posted 1:29 pm on Jan 2, 2014 (gmt 0)

Freshly nabbed nicking without even a glimpse at robots we present;

eNET Inc. OH USA
207.182.128.0 - 207.182.159.255 207.182.128.0/19

207.182.143.nnn - - [01/Jan/2014] "GET /example.htm HTTP/1.1" 200 9869 "-" "niki-bot"

Don't feel bad KeyP:
We've had 5. blocked for years, jam-packed with nasties from Latvia and associated alcoholic regions.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4607411 posted 8:22 pm on Jan 2, 2014 (gmt 0)

We've had 5. blocked for years

ALL of 5? Do you deal in geographically constrained widgets?

RIPE ranges aren't regionally apportioned. You'll find legitimate academics from the UK side by side with Ukrainian robots.

This 327 message thread spans 11 pages: < < 327 ( 1 2 3 4 5 6 7 [8] 9 10 11 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved