stcrim
msg:27739 | 1:38 pm on Oct 16, 2000 (gmt 0) |
Hope, Many people are running into the same slurping situation. You are wise to not want to run robots.txt to slow'er down. At least for us, there has been no benefit, no random pages getting indexed, no improvement in ranking - in fact, it appears slup is just visiting with no purpose (can't be, right?) We have come to ignore the visits and filter for them in the logs. If there's a better suggestion - I too will look forward to it. Steve
|
mark roach
msg:27740 | 2:02 pm on Oct 16, 2000 (gmt 0) |
Sorry to trivialise a serious thread. But when I was at school we used to put small spiders inside the gas nozzles in the science labs. That used to slow them down, they used to walk around dazed for a few seconds before scurrying off. Alternatively you could detour slurp over to my site, I haven't seen him in ages. Perhaps my past cruelties have finally caught up with me :)
|
drbill
msg:27741 | 2:54 pm on Oct 16, 2000 (gmt 0) |
Hope, I have been getting hit about 12000 a day by slurp that is total on my server. I, like stcrim have just ignored him now as it seems to do not good to get excited about him any more.. Maybe one day I can get excited when i see him but not right now. Any one getting listings higher than 100 in one word searchs?
|
Hope
msg:27742 | 5:37 pm on Oct 16, 2000 (gmt 0) |
>>>it appears slup is just visiting with no purpose<<< I can't complain about what slurp is picking up. We are lucky to have new pages added to the index all the time. >>>Alternatively you could detour slurp over to my site, I haven't seen him in ages<<< mark, I would be happy to send slurp your way if I knew how. >>>Any one getting listings higher than 100 in one word searchs?<<< We have lots of listings in the top 10 for one word searches. :)
|
stcrim
msg:27743 | 5:47 pm on Oct 16, 2000 (gmt 0) |
>>We have lots of listings in the top 10 for one word searches. Hope, Is that new pages or pages pre-June or this year??? And are they competitive keywords and phrases??? Steve
|
engine
msg:27744 | 5:49 pm on Oct 16, 2000 (gmt 0) |
>We are lucky to have new pages added to the index all the time.
Hope, are you doing anything other than just providing a link to these "new" pages?
|
Hope
msg:27745 | 9:15 pm on Oct 16, 2000 (gmt 0) |
>>>>Is that new pages or pages pre-June or this year??? And are they competitive keywords and phrases???<<<< The pages in the top 10 are both old and new. I can't say that they are competetive words. >>>>are you doing anything other than just providing a link to these "new" pages? <<<< These pages are just linked from other sections of the site. There is nothing special about them. To be honest, most of these sites only have titles; no description or keywords. They are not optimized in any way. The pages are rich in content and the site is very theme oriented.
|
|
