Forum Moderators: open
I have a database-driven site that's fairly high traffic (over 6M accesses a month) and it's attracted a LOT of weird scripts, bots, Java apps, and bad guys trying to iteratively harvest or walk through my copyrighted database.
I seem to recall reading in this forum that self-promotion here is frowned on, so if you're interested in what my site does, just go to Google and search for my username here (acronym) and hit "I'm Feeling Lucky" :-)
I use Win2k, IIS, and SQL Server 2000 for my site. I develop in ASP, ASP.NET, SQL, VB.NET, etc.
In addition to combing my logs for bad guy UAs, I had to implement an IP tracking database to limit queries on my site by IP address. I'm currently tracking over 3 million IPs of my site's visitors.
I also block or limit a lot of User Agents and would be happy to share my list.
Now that I've introduced myself, I'd like to know if there are other IIS folks here, and if so, is there an equivalent to htaccess in IIS? (I do not have direct control over my IIS Admin stuff.)
Thanks again for all the useful tips and tricks about UAs and I'm really glad to be here.
Cheers,
Mike
Welcome to WebmasterWorld [webmasterworld.com]!
Technically, posting your search term is frowned on, just as posting your own URL is. You can list your URL in your WebmasterWorld user profile, where it will become visible after you have posted a certain number of times. Understand that this rule is in place to prevent one-time posters coming here to drop their URL for promotional purposes - even though that doesn't work (yawn, we've seen it all), it clutters up the forum.
With that out of the way, I believe that what you are looking for in IIS is a file called browsecap.ini - I'm not sure, because I host on Apache, but it's been discussed here before. Try the site search link at the top of the page, using "IIS" and "block bad bots" or "spiders."
Hope this helps!
Jim
Oops, sorry about that.
Is there an etiquette FAQ for general posting rules and also a tedhnical FAQ for the Spider ID Forum?
Thanks,
Mike
Yes, see the "Welcome" link I posted above for a general introduction.
The links at the top and bottom of each page are quite useful as well. Try the site search and the library for technical summaries, read the Terms of Service below to avoid having your posts edited.
Jim
just found this one, don't know whether it works because I'm on apache.
evolvedcode.net/content/code_crawlerfilter/
jan
anyway, why don't you think about moving your site on a apache system. iis is not that hard rocking at all.;)
I prefer Apache too, but try this thread [webmasterworld.com] for some more info. I found several others searching WebmasterWorld with "iis block ban" as well.
Jim
Yes we can use a three tiered approach with IIS with global.asa, ini's, and ISAPI filters. It will work just as well as the apache stuff, just noone talks about it. :)
I wish there was more discussion of these issues on MS servers - I might easily end up working on an existing IIS or Win2K account that needs some "security enhancements"... Nice to know where to start!
Jim
if UA = "bad guy UA" Then
response.redirect("go_to_hell.asp")
end if
It's actually a little more sophisticated than that, and I would actually like to put the bad UA list in a database, but I've been trying to avoid having to use SQL Server resources just to look up and block a bad guy.
Never thought to do it at the global.asa level. I'd like to do some benchmarking though as something about that seems counterintuitive to me.
So you folks doing your blocking at global.asa don't see any performance issues that don't scale well?
Mike
