lucy24 - 11:13 pm on Aug 24, 2013 (gmt 0)
Can anyone shed any light on ExactSeek? Forums search doesn't turn up anything more recent than 2008; most discussion is much earlier (thread numbers in the lower 4 digits, meaning 2003 or so). I get the impression they're a pay-for-inclusion, by-request-only search engine... but if so, what's their robot doing on my site? I've definitely never had any interaction with them, but they've been crawling sporadically for months, with a previous spurt of activity last year. (Do they take the winter off? Where do they go?)
Viewed purely as a robot they're perfectly well behaved: always ask for robots.txt and appear to obey it. ("Appear to" = none of my roboted-out directories are directly linked from the front page, and they never go beyond the first link, except for one directory that for some reason seems to interest them.)
What do they want, and will it do any harm to let them have it?