AskJeeves visited my site yesterday - to the best of my knowledge, the first time the bot was ever there. It got robots.txt and the home page.
Today it was back, but it requested a whole bunch of obsolete pages, each of which returned a 300 code (they used to be .htm, are now .php), but the bot didn't go for the new pages. I'm scratching my head as to what happened - there are no links to those old pages to be found in ANY document in my webspace.
I did notice that the request for the homepage came through as HTTP/1.0, and the number of bytes was about 50 lower than requests for the same page as HTTP/1.1 (but I think this is normal? I don't understand the difference).
Any ideas as to what could be going on?
