I'd like to know if there's somewhere I could get to get a very precise and very regularly updated list of IP addresses of the various crawlers/spiders... I know you can't talk about specific programs, but if someone could PM me I'd appreciate it. Or, if there's another source, perhaps a website or something, could you tell me please?
Thanks... I'm trying to find something reliable that keeps up-to-date with them.
Most cloaking scripts these days include a regularly updated list of spiders/IP's. If you are using a script, ask the admin if they have a URL for you. If you are not using any script, just search on Google for it.