Forum Moderators: open
I just noticed that the Search Engine Identification forum is being depricated... Can I ask what the rational behind this is? I find this one of the most valuable parts of WebmasterWorld...
J
Oh, he also closed the Deprecated - Search Engine Spider Identification forum.
Pendanticist.
Basically, the problem is that of posting IP addresses publically. Would you want your ip address posted on the highest traffic webmaster site on the net? Think about it...
We are discussing alternatives.
Again, interested in how it pans out.
You say you're concerned about people posting IPs. That info is needed to determine if the bot should be banned by IP or UA as well as figuring out whether it really belongs to a SE or is an imposter. Without sharing info on these, people will just blindly copy the "perfect ban" list as being gospel not having a clue what any of it means, or that they may never see the majority of those on that list, or wonder why one wasn't stopped by an incorrect directive.
Now, if someone out there wants to write a bot then fine. But there is a way of writing a bot that is socially responsible. Have a meaningful UA, don't grab too many pages at a time, take notice of my robots.txt file. You don't do that then you get banned.
Get banned by me at least. If your IP has been posted then you *may* banned by others. You may get banned because you are from the wrong country. You may get banned because you are the wrong type of bot. You may get banned becuase you are a malicious bot. That decision is up to each webmaster.
And that's what I'd argue for keeping the spiders forum. By exposing these IP's we can try and generate social pressure to stop malicious bots. And by giving out the IP we allow webmasters to force these bots out technically.
Just because an IP or bot was bad on one site did not mean it was bad in mine. I also seem to be banning more UA's (or certain words) than IP's anyways (as the bot trap points those out to me) and this seems to be effective, (unless a certain ip is bothersome).
I think the most beneficial thing for people is to learn 'how' to ban IP's and bots, (how to make a botrap/read logfiles/determine if a visitor is misbehaving/where to look for the UA etc.etc.), rather than "what" to ban, that's all we really need to know.
As said previously, it's really up to the individual webmaster as he studies his logfiles.
If the search engine identification forum was causing serious problems for Webmasterworld, then I'd rather it be closed! I'm sure this will take nothing away from this site, only add to it.
Thank you for providing us with Webmasterworld.
It's one of the very few worthwhile and valuable sites on the net.
I'd rather this could be discussed so that a more proactive decision could be determined. Deciding not to talk about (potentially) bad bots is just going to drive the discussion away to somewhere else. Also it is fine for people who have the resources to find malicious bots, but what about the smaller guys?
So are malicious bots going to be allowed to be discussed in the robots.txt forum? Are bots that aren't from the main searchengines going to be discussed? If they are then I don't see a problem with the old forum, and if they aren't then I must say I'd be very disappointed...
A few dufus heads aren't posting bots. They are posting competitors or people they want attacked ip's.
According to legal advice, posting someones ip may be determined to be personally identifiable information and against the law in some countries. aka: red flag houston - legal alert - stop the presses - end of story.
So will posting useragents be allowed? Its very annoying that some people are posting misleading information for personal gain or for some stupid personal vendetta...
and so people will of course believe me and Ban all the hits coming from 55.676.22.42
Sid
