Genieo Web Filter

[keyplyr]

UA: Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html
robots.txt: no

Scrapes Twitter for links.
Comes from various IP addresses.
UA can also display during visit at varied times.

Seems be a security or parental filter plugged into the user's browser as well as a stand alone crawler building a database.

I've been blocking it for 3 weeks. Anyone have additional info?

[keyplyr]

Notice the missing closing parenthesis.

[lucy24]

Logs say:

Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html

(with, yup, an odd number of parentheses)

Site says:

Mozilla/5.0 (compatible; Genieo/x.x http://www.genieo.com/webfilter.html)

and

We at Genieo, [**sic comma] design our client software to access only pages which are reasonably likely to interest the user and appear on their homepage.

followed by a bunch of blahblah which interests me much less than the question of how they prevented my browser from showing a horizontal scroll bar even when their text didn't fit. But I guess that's for a different forum.

Picked up my home page, which nobody ever goes to, so no idea what they're up to.

whois tells me only that the most recent IP is from a /12 range belonging to qwest-- which I already knew, because there are humans in there.

[keyplyr]

Hits me about 30 times a day, always just HTML, always 403. Pretty stupid.

[fips]

[mugo.ca...]