Forum Moderators: open
robots.txt? NO
Social network crawler; search site; toolbar apps. IP address -- 216.24.139.144 -- is oneriot.com site address (robtex [robtex.com]) because people search from there. However --
It seems odd anyone would be searching for/finding us 'socially' as often as this UA is hitting on us. No referer info means no helpful keywords and also leaves open the possibility that something is crawling other than social sites, on its own...
Anyway.
Previously, from 216.24.131. a.k.a. server farm viawest.net and/or "Getmedium.com" / Medium, Inc. / Me.dium, Inc / "dium.com":
216-24-131-119.viawest.net
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 Me.dium/1.0 (http://me.dium.com)
216-24-131-152.viawest.net
LijitSpider/Nutch-0.9 (Reports crawler; http://www.lijit.com/robot/crawler; info(a)lijit(d)com)
Related, via UA and ViaWest:
flx1-ppp4n.lvdi.net
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0 Me.dium/1.0 (http://me.dium.com)
Last but not least, also from robtext -- they've got the tpyos and mispelings covered:
getmedium.com
oneriiot.com
oneeriot.com
onreiot.com
oneirot.com
onerot.com
onerriot.com
onerioot.com
onoriot.com