homepage Welcome to WebmasterWorld Guest from 54.145.183.169
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Level 3 Communications
Anyone getting hits from 8.11.2.xx?
Bewenched

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4514000 posted 1:09 am on Oct 31, 2012 (gmt 0)

They seem to come by every evening, but they aren't grabbing pages and I dont have them blocked.... for now.


8.11.2.10

I have the worlds crappiest log tool I'd literally have to parse the log myself to find out the user agent.

 

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4514000 posted 6:29 am on Oct 31, 2012 (gmt 0)

I have the worlds crappiest log tool

Uhm... So get a better one :P

I'd literally have to parse the log myself to find out the user agent.

Find in raw logs:
^8\.11\.2\.10\b

grab all hits, pull into a placeholder file.

replace
^[\d.]+ - - \[\S+ -\d\d00\] "GET \S+ HTTP/1\.[01]" \d+ \d+ "[^"]+" "(.+)" ?\n
with
$1\n

There's your UA.

(Haven't personally met your guys. My notes show assorted AV-type things, a crummy free-hosting place that's attractive to hotlinkers, a minor blog site and Yahoo Mobile. All of these are in the 8.12 - 8.28 range. No idea what goes on in the rest of the neighborhood.)

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4514000 posted 7:11 am on Oct 31, 2012 (gmt 0)

Level3 is an old thorn.

A few of us have all or nearly all their ranged denied.

The Class A's (4 & 8) are almost elementary, despite being used by many commercial networks.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4514000 posted 8:24 am on Oct 31, 2012 (gmt 0)

8, sure. But most of the 4's I've met are human-- including at least two WebmasterWorld members ;)

Bewenched

WebmasterWorld Senior Member 5+ Year Member



 
Msg#: 4514000 posted 6:53 pm on Oct 31, 2012 (gmt 0)

Finally found a decent free one, not too shabby either.

Wilderness, could you share the class A's ranges with us.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4514000 posted 7:17 pm on Oct 31, 2012 (gmt 0)

Bewenched,
I opended up some of these ranges in Feb 2012, however previously I had them all denied (there may even be more today):

166.90.0.0 - 166.90.255.25
171.75.0.0 - 171.75.255.255
192.231.42.0 - 192.231.42.255
199.92.0.0 - 199.95.255.255
204.164.0.0 - 204.167.255.255
206.15.0.0 - 206.15.31.255
206.32.0.0 - 206.35.255.255
207.227.0.0 - 207.227.255.255
207.90.128.0 - 207.90.191.255
209.0.0.0 - 209.0.255.255
4.0.0.0 - 4.255.255.255
63.208.0.0 - 63.215.255.255
64.152.0.0 - 64.159.255.255
67.24.0.0 - 67.31.255.255
8.0.0.0 - 8.255.255.255

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4514000 posted 10:10 pm on Oct 31, 2012 (gmt 0)

I have the range 8.11.2.0/24 listed as shopwiki, with 8.11.2.9 and 8.11.2.10 as shopwiki bot. This was back in March 2010. Robtex cnet shows 9, 10 as pep2 and pep1 dot shopwiki so probably still valid.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4514000 posted 10:20 pm on Oct 31, 2012 (gmt 0)

Courtesty SEE's block-edit function:

Deny from 4 (isn't that fun? "Deny from 4.0.0.0/8" if the bare form makes you nervous)
Deny from 8 (ditto)
Deny from 63.208.0.0/13
Deny from 64.152.0.0/13
Deny from 67.24.0.0/13
Deny from 166.90 (assuming the final "25" was a typo for "255" since the alternative doesn't bear contemplating)
Deny from 171.75
Deny from 192.231.42
Deny from 199.92
Deny from 204.164
Deny from 206.15.0.0/19
Deny from 206.32.0.0/14
Deny from 207.227
Deny from 207.90.128.0/18
Deny from 209.0

Or, if you prefer,
RewriteCond %{REMOTE_HOST} ^166\.90 [OR]
et cetera

or conversely (with implied AND)
RewriteCond %{REMOTE_HOST} !^166\.90
et cetera

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4514000 posted 12:19 am on Nov 1, 2012 (gmt 0)

(assuming the final "25" was a typo for "255" since the alternative doesn't bear contemplating)


I never make typo's ;)

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4514000 posted 12:44 am on Nov 1, 2012 (gmt 0)

Me neither:
Courtesty SEE's block-edit function

Ahem.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved