| What is Slurp up to?
Getting very carried away... |
Machiavelli
msg:24441 | 2:21 pm on Jan 22, 2001 (gmt 0) |
I have a few free web sites for testing purposes/playing around for which I have access to the web logs. There is nothing of much interest on these, but I do seem to notice that Slurp seems to visit quite a bit (normally the one which calls itself: Mozilla [something or other] (Slurp [something or other]) I haven't submitted these pages to Ink or PT, they do not appear on a domain:domain.com search, and yet they have been coming around for months. Is slurp just bored?
|
msgraph
msg:24442 | 2:36 pm on Jan 22, 2001 (gmt 0) |
If it has Slurp/si in the UA then it's probably checking for updates on the page. Have you updated the content in them at all lately? If you make some updates and this spider continues to come by, they will probably throw it back into their index.
|
|
|
