homepage Welcome to WebmasterWorld Guest from 54.161.191.154
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Fieldtech?
lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4643761 posted 10:13 pm on Feb 9, 2014 (gmt 0)

Has anyone got the remotest idea who these people are?

IP: 38.99.82.abc
They appear to own the whole /24, though all crawling to date is from the narrower .192/26 subsector.

UA: Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 6.0)
But see below.

Far as I can make out they belong to an outfit called Fieldtech. The one in Idaho, not to be confused with the (probably) several dozen other companies with the same name. They ask for and appear to obey robots.txt, and generally crawl at a leisurely pace. ("Appear to" = they've only gone two steps from the front page.) I've met them twice: once in the first half of January, and again a few days ago.

Mystifying quirk:
Alongside their ordinary crawling, each visit has included a block of exactly 18 (eighteen) requests for the same interior page. For these they used a different UA
Java/1.6.0_33
which got them blocked. Eighteen times in rapid succession.

 

Angonasec

10+ Year Member



 
Msg#: 4643761 posted 3:14 pm on Feb 10, 2014 (gmt 0)

Any PSI/Cogent CIDR is unwelcome on our sites.
Currently have 6 ranges blocked, but 38.99. hasn't dared knock on our doors yet...

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4643761 posted 9:30 pm on Feb 10, 2014 (gmt 0)

I have the range 38.96.0.0 - 38.127.255.255 blocked as "servers".

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4643761 posted 1:33 am on Feb 11, 2014 (gmt 0)

I have the range 38.96.0.0 - 38.127.255.255 blocked as "servers".

I could never do that; I see too many humans from this area. They tend to be Canadian schools,* probably sub-sub-subletting their /27 and /30 slivers.


* I'm pretty sure it was in this area that I first encountered "Separate" as a noun.

motorhaven

10+ Year Member



 
Msg#: 4643761 posted 4:49 pm on Feb 12, 2014 (gmt 0)

Dstiles, I have a boatload of humans coming through there, especially 38.100. and 38.108.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4643761 posted 9:49 pm on Feb 12, 2014 (gmt 0)

I need the sub-ranges identified. The current identity (via arin) is useless. Small sub-ranges of /29 or /25 suggest businesses rather than individuals, and a lot of businesses either do not use our services or abuse them (cf comcast).

Also bear in mind that I am in the UK and get so many bad hits from US that I'd really like to block the whole country; pity about my customers, who won't let me do that. :(

If anyone has a reasonable breakdown of the range I will consider opening up some of it, but the holes will need to be at least /20.

wilderness

WebmasterWorld Senior Member wilderness us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4643761 posted 1:56 am on Feb 13, 2014 (gmt 0)

PSI/Cognet is assigned the entire Class A.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4643761 posted 3:52 pm on Feb 13, 2014 (gmt 0)

But so are several other providers. Most of them give breakdows via DNS assignements. Cogent seem too lazy to do this - or they consider the whole /11 to be one big mish-mash of business/server/whatever, which drops them into a hack blacklist anyway.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4643761 posted 10:58 pm on Feb 13, 2014 (gmt 0)

especially 38.100. and 38.108.

Heh. I think at one time I had 38.100 blocked until I realized there are humans in there. Most of mine seem to be from 104 and 107-- but the slivers are so tiny, this really is random.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved