Welcome to WebmasterWorld Guest from 54.227.1.130

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

LightningDownload/1.0.1

Should this be labeled a good bot or a bad bot?

     

Pushycat

3:29 pm on Aug 15, 2002 (gmt 0)

10+ Year Member



The above is a product lets users select stuff to download from the web and a search turned up no mention of it here. The URL for the product is the same as the user agent name.

Should the fact it does not read robots.txt make it a bad bot that I exclude?

Or does the fact that the maximum requests it made in a 1 minute period were 22 with most of them at around 6 or 7 a minute make it a good bot even though it took stuff from folders that are excluded in robots.txt for all bots?

I'm trying to decide whether to ban it or not. Thanks in advance for your comments.

Knowles

3:37 pm on Aug 15, 2002 (gmt 0)

10+ Year Member



If you dont want it downloaded and taken then it would be bad. Most of these products dont have any real use other than pulling the website for either offline usage or to take the code out of it. I would ban it if it bothers you. If it doesnt dont worry about it.
 

Featured Threads

Hot Threads This Week

Hot Threads This Month