Welcome to WebmasterWorld Guest from 54.159.250.110

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Rainspider/1.0
3 Steph 8:38 am Jul 2, 2001
Are these spiders?
7 terryl 2:20 am Jul 1, 2001
$%^$%$# Wget!!
Strange occurance...
4 icehousedesigns 12:03 pm Jun 30, 2001
mystery
2 roscoepico 4:40 pm Jun 29, 2001
SliderSearch_v1
Ever hear of him?
3 awoyo 7:50 pm Jun 28, 2001
Clcpnpzepkl - 65.2.132.203
Another @Home Jockey
1 jimmykav 11:23 am Jun 28, 2001
Google Image Spider
What big i's it has...
3 Josk 8:56 am Jun 28, 2001
lwp-trivial?
4 Brett_Tabke 11:36 pm Jun 26, 2001
Hits from bigip1-snat.sv.av.com ( 209.73.164.50)
Altavista new bot ?
4 bfleisch 11:38 pm Jun 25, 2001
Heads up (as toolman would say)
3 Froggyman 6:32 pm Jun 25, 2001
Is this an Alexa spider?
3 toolman 4:08 pm Jun 25, 2001
Disney.com Spidering
lwp-trivial/1.28
2 Froggyman 1:09 pm Jun 25, 2001
Kjjwro
3 jimmykav 12:48 pm Jun 25, 2001
tivra spider
2 roryfrost 12:29 am Jun 25, 2001
Mercator
v1-2jg
3 startup 5:17 am Jun 23, 2001
Spiders and counter stats
3 Outtatheblue 5:05 pm Jun 22, 2001
Atom Indexer v1.1a
217.11.167.34
6 skirril 3:43 pm Jun 19, 2001
This is new...
Ink UK?
2 toolman 9:54 am Jun 19, 2001
Anyone know who this is?
209.67.206.133
3 toolman 8:16 am Jun 17, 2001
MSN what is the useragent
2 zipup 9:02 pm Jun 15, 2001
List your banned IP's / domains and why
7 icehousedesigns 1:06 pm Jun 14, 2001
Teoma Spider IP list
where can I find one?
5 Travoli 1:04 pm Jun 14, 2001
Crescent Internet ToolPak HTTP OLE Control v.1.0
Should I ban it in my robots.txt file?
2 merlin78 5:26 am Jun 14, 2001
Inktomi hitting pages over and over
what the heck?
5 Travoli 1:04 pm Jun 13, 2001
AOL UAbut different IP
4 toolman 8:08 pm Jun 12, 2001