Ask Jeeves just grabbing robots.txt and nothing else

Forum Moderators: open

Message Too Old, No Replies

Ask Jeeves just grabbing robots.txt and nothing else

msgraph

3:24 pm on Jun 26, 2001 (gmt 0)

Anyone else seeing the AskJeeves/DH spider hit their sites and only ask for the robots.txt and nothing else?

64.55.148.50
Mozilla/2.0 (compatible; Ask Jeeves)

jeremy goodrich

3:08 pm on Jul 11, 2001 (gmt 0)

Any more news on their spider, msgraph?

I still haven't gotten 'em to show up in logs that I am watching. Does ask jeeves/direct hit's spider still do this, or have they gone deeper into your site(s) by now?

msgraph

4:44 pm on Jul 11, 2001 (gmt 0)

I've been watching 'ol Jeeves and he's been coming back from time to time. It's pretty weird, however, because one day it'll grab the robots.txt and the root index page. Then the next day come back just to grab the robots file. I've seen this happen on a large number of domains.

I haven't seen it grab anything in the sub-level, nor have I seen much of any results from an ASK or DH listing. But also, I haven't focused on improving any listings either.