homepage Welcome to WebmasterWorld Guest from 54.163.72.86
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Zvelo?
bakedjake




msg:4433679
 8:57 pm on Mar 26, 2012 (gmt 0)

Always from 216.38.216.101 (reverse: gate.zvelo.com)
UA is typically

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_1) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.215 Safari/535.1

but sometimes

Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)

Definitely a bot. It requests erroneous URLs the same way, every time, and then proceeds to deconstruct the URL:

GET /dir/page [no ref]
GET /dir [no ref]
GET / [no ref]
GET / [ref: http://www.google.com/trends/hottrends]

 

wilderness




msg:4433744
 12:49 am on Mar 27, 2012 (gmt 0)

There was a content filter that ran the Wget software in 2002 from the same backbone provider, however a different IP block.

During 2008 this content filter made the transition to a masked UA.

I've had the this 216 range denied since 2009 for mysterious activity.

Were webmasters capable of subnet searches, than there'd be less innocents service-less. (Suppose I could not be so extreme and make the denied range smaller).

Two things would need to happen:
1) More webmasters would need to care and be aware of the traffic that passes through their doors.
2) Backbones would need to ID the subnets.

keyplyr




msg:4433778
 4:15 am on Mar 27, 2012 (gmt 0)

viawest.net is a colo and managed server farm

216.38.192.0 - 216.38.223.255
216.38.192.0/19

I also have their other ranges blocked:

216.24.128.0 - 216.24.159.255
216.24.128.0/19

Anyone have more?

wilderness




msg:4433799
 5:48 am on Mar 27, 2012 (gmt 0)

Anyone have more?


Your beckon call ;)
204.246.128.0 - 204.246.151.255
216.87.64.0 - 216.87.95.255

I'm sure there's more.

lucy24




msg:4433804
 6:55 am on Mar 27, 2012 (gmt 0)

128-151? Really?
:: detour to whois ::
Ugh. Really. I thought that kind of thing was comcast's specialty.

204.246.128.0/20 AND 204.246.144.0/21

keyplyr




msg:4433806
 7:08 am on Mar 27, 2012 (gmt 0)

Thanks Don

wilderness




msg:4433907
 1:46 pm on Mar 27, 2012 (gmt 0)

So how do I define and translate this?

2600:3000:: - 2600:3007:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF


Would you just focus on the 3000-3007?
Will mod_Rewrite and all the others accept IPv6 Ranges?

keyplyr




msg:4434058
 6:43 pm on Mar 27, 2012 (gmt 0)

Just a FYI - Your server needs to be IP6 for your code to work.

MxAngel




msg:4434127
 9:35 pm on Mar 27, 2012 (gmt 0)

viawest ranges:
[whois.arin.net...]


VIAWEST-BLK-12 (NET-216-46-160-0-1) 216.46.160.0 - 216.46.191.255
VIAWEST-BLK (NET-64-90-192-0-1) 64.90.192.0 - 64.90.207.255
VIAWEST-BLK-14 (NET-64-78-176-0-1) 64.78.176.0 - 64.78.183.255
VIAWEST-BLK-15 (NET-64-78-184-0-1) 64.78.184.0 - 64.78.199.255
VIAWEST-BLK-13 (NET-64-78-144-0-1) 64.78.144.0 - 64.78.159.255
VIAWEST-BLK-16 (NET-64-78-224-0-1) 64.78.224.0 - 64.78.239.255
VIAWEST-BLK-2 (NET-216-38-192-0-1) 216.38.192.0 - 216.38.223.255
VIAWEST-BLK-1 (NET-216-87-64-0-1) 216.87.64.0 - 216.87.95.255
VIAWEST-BLK-5 (NET-216-150-192-0-1) 216.150.192.0 - 216.150.223.255
VIAWEST-BLK-3 (NET-216-58-128-0-1) 216.58.128.0 - 216.58.175.255
VIAWEST-BLK-6 (NET-206-71-64-0-1) 206.71.64.0 - 206.71.95.255
VIAWEST-BLK-7 (NET-204-246-128-0-1) 204.246.128.0 - 204.246.151.255
VIAWEST-BLK-9 (NET-216-24-128-0-1) 216.24.128.0 - 216.24.159.255
VINS-BLK-9 (NET-66-114-96-0-1) 66.114.96.0 - 66.114.127.255
VIAWEST-BLK-10 (NET-216-241-96-0-1) 216.241.96.0 - 216.241.111.255
VIAWEST-BLK-11 (NET-209-170-192-0-1) 209.170.192.0 - 209.170.223.255
VINS-BLK-7 (NET-207-189-96-0-1) 207.189.96.0 - 207.189.127.255
VINS-BLK-2 (NET-216-241-160-0-1) 216.241.160.0 - 216.241.191.255
VINS-BLK-1 (NET-64-58-224-0-1) 64.58.224.0 - 64.58.239.255
VINS-BLK-3 (NET-66-51-0-0-1) 66.51.0.0 - 66.51.31.255
VINS-BLK-4 (NET-66-97-128-0-1) 66.97.128.0 - 66.97.143.255
VIAWEST-BLK-18 (NET-216-250-128-0-1) 216.250.128.0 - 216.250.143.255
VIAWEST-BLK-19 (NET-66-133-96-0-1) 66.133.96.0 - 66.133.127.255
VINS-BLK-8 (NET-216-145-112-0-1) 216.145.112.0 - 216.145.127.255
VIAWEST-BLK-20 (NET-67-208-128-0-1) 67.208.128.0 - 67.208.143.255
VIAWEST-BLK-17 (NET-74-63-128-0-1) 74.63.128.0 - 74.63.191.255
VIAWEST-IPV6-BLK-1 (NET6-2600-3000-1) 2600:3000:: - 2600:3007:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF
VIAWEST-BLK-21 (NET-209-236-96-0-1) 209.236.96.0 - 209.236.111.255

[edited by: MxAngel at 9:36 pm (utc) on Mar 27, 2012]

dstiles




msg:4434128
 9:35 pm on Mar 27, 2012 (gmt 0)

My Viawest list:

74.63.128.0 - 74.63.191.255
204.246.128.0 - 204.246.159.255
207.189.96.0 - 207.189.127.255
216.24.128.0 - 216.24.159.255
216.38.192.0 - 216.38.223.255
216.87.64.0 - 216.87.95.255
16.150.192.0 - 216.150.223.255

keyplyr




msg:4434212
 1:04 am on Mar 28, 2012 (gmt 0)


@dsriles

Sure that last one (16.150.192.0 - 216.150.223.255) is Viawest? I get HP.

MxAngel




msg:4434297
 8:31 am on Mar 28, 2012 (gmt 0)

@keyplyr,

Last one should read: 216.150.192.0 - 216.150.223.255 and not 16.150.192.0 - 216.150.223.255 ;)

keyplyr




msg:4434318
 10:12 am on Mar 28, 2012 (gmt 0)

Oh yeah... don't know why I didn't see that - ha

dstiles




msg:4434544
 8:59 pm on Mar 28, 2012 (gmt 0)

Sorry, folks - too delet-key happy. :(

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved