homepage Welcome to WebmasterWorld Guest from 54.242.126.9
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Icap-iod
from Internap
keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4678587 posted 9:49 pm on Jun 9, 2014 (gmt 0)

Caught a dozen HEAD requests for image files from the UA: ICAP-IOD
No robots.txt request. All these files are noindexed via X-Robots-Tag (of course the file header would need to be read for the tag to be found.)

ICAP-IOD making requests from Internap:
216.223.24.0 - 216.223.27.255
216.223.24.0/22

Regardless of intent, I blocked them via UA and IP range.

Other Internap ranges I have blocked are:

63.251.0.0 - 63.251.255.255
63.251.0.0/16

66.150.0.0 - 66.151.255.255
66.150.0.0/15

69.25.0.0 - 69.25.255.255
69.25.0.0/16

70.42.0.0 - 70.42.255.255
70.42.0.0/16


74.217.0.0 - 74.217.255.255
74.217.0.0/16


95.172.68.0 - 95.172.71.255
95.172.68.0/22


208.64.56.0 - 208.64.63.255
208.64.56.0/21

216.52.0.0 - 216.52.255.255
216.52.0.0/16

 

not2easy

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



 
Msg#: 4678587 posted 4:36 am on Jun 10, 2014 (gmt 0)

Last June and October I found various HEAD requests from internap , actually a "Private Customer" INAP-SJE-PALOALTOTECHOPS
64.74.215.0 - 64.74.215.255
in this net
Internap Network Services Corporation PNAP-SEA-BLOCK4
64.74.0.0 - 64.74.255.255

Not the same UA though, it was "'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)'" (yes, with the extra single quote).

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4678587 posted 5:03 am on Jun 10, 2014 (gmt 0)


Thanks for the info and additional range.

not2easy

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



 
Msg#: 4678587 posted 5:54 am on Jun 10, 2014 (gmt 0)

I have more ranges, they don't involve this UA and they aren't noted as "HEAD" requests so I didn't include them all before.

This list is from last January here in the forum:
Internap:

63.251.0.0 - 63.251.255.255
64.7.192.0 - 64.7.223.255
64.74.0.0 - 64.74.255.255
64.94.0.0 - 64.95.255.255
66.150.0.0 - 66.151.255.255
67.63.48.0 - 67.63.63.255
69.25.0.0 - 69.25.255.255
70.42.0.0 - 70.42.255.255
72.5.0.0 - 72.5.255.255
74.201.0.0 - 74.201.255.255
74.217.0.0 - 74.217.255.255
75.98.0.0 - 75.98.95.255
77.242.192.0 - 77.242.207.255
95.172.64.0 - 95.172.95.255
207.174.0.0 - 207.174.255.255
212.118.224.0 - 212.118.255.255
216.52.0.0 - 216.52.255.255
216.133.224.0 - 216.133.255.255

And a few more from my big list and some log notes:
Internap Network Services Corporation (PNAP) 206.191.128.0 - 206.191.191.255
206.191.128.0/18
INTERNAP-SEA (NET-206-253-192-0-1) 206.253.192.0 - 206.253.223.255
206.253.192.0/19

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4678587 posted 8:42 am on Jun 10, 2014 (gmt 0)

Great, thanks again

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4678587 posted 5:05 pm on Jun 10, 2014 (gmt 0)

yes, with the extra single quote

I see those superfluous quotation marks now and then. (Query: Does there exist any legitimate UA that either begins or ends with a quotation mark?) It now occurs to me to wonder if the originator is copy-and-pasting from some standard UA list, and accidentally included the enclosing quotes.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved