Toggle navigation
Dashboard
Dashboard Tracker
Add / Edit My Forums
System Announcements
Discussion
Active Thread Feed
Top Threads Today
Top Threads This Week
Top Threads Library
Go!
View All
View All
Login
Logout
Home
Forums Index
General SEO Issues
/
Crawler, Spider, and User Agent ID
3:28 pm May 21, 2026
This page requires javascript.
Forum Moderators:
open
Message Too Old, No Replies
Definitive list of spiders.
lloyd
Msg#:400341
2:40 pm on Oct 25, 2004
(gmt 0)
Is there an up to date definitive list of search engine spiders out there. Either on this forum which I can't find or on a specialist website.
volatilegx
Msg#:400342
3:03 pm on Oct 25, 2004
(gmt 0)
See [
webmasterworld.com
...]
lloyd
Msg#:400343
3:17 pm on Oct 25, 2004
(gmt 0)
2002 is now old in search engine terms. Is there anything more recent.
wilderness
Msg#:400344
10:54 pm on Oct 25, 2004
(gmt 0)
bulls collated bot list (last updated on August 20th) is the most recent.
[
webmasterworld.com
...]
Page #2 Message #'s 16 and 17.
volatilegx
Msg#:400345
4:14 pm on Oct 26, 2004
(gmt 0)
lloyd, the post I mentioned is old, but the list it points to is still being updated.
Join The Conversation
Register
For Free! -
Become a
Pro Member
!
See forum categories
-
Enter the Forum
Moderators and Top Contributors
Moderator List
| Top Contributors:
This Week
,
This Month
,
Apr
,
Mar
,
Archive
,
Top 100 All Time
,
Top Voted Members
Hot Threads This Week
May 2026 Google Search Observations
AdSense Earnings and Observations - May 2026
AI visibility audit
Adsense / Personal Manager / Support
Hacker spoofing as GoogleBot
Page indexed but live url test gets 500 error
Link collaboration strategies
in case you missed it .
Microsoft, AS8075. Google, AS396982.
Bad actor claiming to be OpenAi
Home
Forums Index
General SEO Issues
/
Crawler, Spider, and User Agent ID
3:28 pm May 21, 2026