Welcome to WebmasterWorld Guest from 54.91.16.95

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Google hardly!
just a heads up
2 wilderness 5:53 pm Jul 3, 2003
tovero
6 wilderness 6:13 am Jul 3, 2003
Netscape/PICgrabber
4 keyplyr 12:23 am Jul 3, 2003
Zao/0.1
Another resurrected spyder?
14 pendanticist 10:58 pm Jul 2, 2003
Univ of Toyko
Just a heads up
2 wilderness 10:53 pm Jul 2, 2003
Firefly
2 WebJoe 11:59 am Jul 2, 2003
New Excite bot?
Similar IP but no ID
3 too_much_information 12:09 pm Jul 1, 2003
python-urllib from another google server
now it comes from another google machine
1 pontifex 10:48 am Jul 1, 2003
grub and ZyBorg
what's going on?
8 claus 8:17 am Jul 1, 2003
212.190.252.64.snet.net - any ideas?
it's a spider for sure, but what is it?
3 claus 7:16 am Jul 1, 2003
203.219.236.102
Who is this?
14 Stefan 12:04 am Jul 1, 2003
Cxhttp
New UA to me
3 MarieC 11:28 pm Jun 30, 2003
AltaVista Robots
What are they called
2 lloyd 7:29 pm Jun 30, 2003
Which spiders do you want crawling your site?
9 ktd121 5:32 pm Jun 30, 2003
Two Bots, 1/2 a world apart- Same Requests
This is weird!
3 carfac 4:20 am Jun 30, 2003
WebGather 3.0
ignores robot.txt
2 frontpage 6:05 am Jun 29, 2003
Mac Finder 1.0.38 and 209.115.232.35
heads up
1 coyote 9:49 pm Jun 28, 2003
217.155.46.193
heads-up
4 wilderness 10:35 pm Jun 27, 2003
168.71.5.202
and a loopy Roadrunner
2 coyote 2:05 pm Jun 27, 2003
Mozilla/8.0
stupid spammer and other bots
4 coyote 11:16 pm Jun 26, 2003
SiteWinder
heads-up
3 wilderness 11:01 pm Jun 26, 2003
216.54.32.154
heads-up
1 wilderness 10:20 pm Jun 26, 2003
MicrosoftPrototypeCrawler - Is it new?
It came to my site.
9 IITian 4:46 pm Jun 26, 2003
dbsmarketing.net & infomaxinc.com
looks like a couple of harvesters
1 jazzguy 7:46 am Jun 26, 2003
MarkWatch/1.0 snoopbot
another IP to add to the block list
2 nativenewyorker 11:13 pm Jun 25, 2003

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members