Reputable Spider lists? - Crawler, Spider, and User Agent ID forum at WebmasterWorld - WebmasterWorld

Forum Moderators: open

Message Too Old, No Replies

Reputable Spider lists?

Trying to find a reputable spider list... what do you recommend?

profitpuppy

11:21 pm on Sep 14, 2005 (gmt 0)

10+ Year Member

Hi, have been looking at my log files and trying to work out which spiders are what and why some bots like some of my sites more than others.

Anyone know of a good spider list? free or otherwise?

volatilegx

8:09 pm on Sep 15, 2005 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

[webmasterworld.com...]

wilderness

12:41 am on Sep 20, 2005 (gmt 0)

WebmasterWorld Senior Member

10+ Year Member

Top Contributors Of The Month

[webmasterworld.com...]