bloglines

crawler?

[Bewenched]

65.214.44.29 (Unknown)4/8/2008 21:31/robots.txt512200 - OK

Gives No Referrer
Gives no user agent.
grabbed the robots.txt

resolves to
crawler.bloglines.com

[Ocean10000]

Bloglines is a FREE online service for searching, subscribing, creating and sharing news feeds, blogs and rich web content.

Question for you. Do you have an RSS feed someplace on your site?

It will download existing RSS feeds, and look for new ones to add to its list. From my reading of there site, a member of there site will subscribe to one of your feeds, so it can be viewed by there service. It does check robots.txt. But other then that I do not know. I block it since it does not supply a User-Agent when reading Robots.txt.

[edited by: Ocean10000 at 5:32 am (utc) on April 10, 2008]

[Bewenched]

Nope .. it asked for my robots.txt file

[incrediBILL]

The crawler.bloglines.com is owned by and used by Ask.com for all sorts of tasks including making screen shots that show up in Asks SERPs.

[edited by: incrediBILL at 8:40 am (utc) on April 11, 2008]

[Bewenched]

Here's the UA string it created the most recent time it came through.

Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a1) Gecko/20070308 Minefield/3.0a1