Welcome to WebmasterWorld Guest from 54.167.46.29

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
Szukacz/1.5
Polish search engine spider
3 Finder 6:46 pm Oct 26, 2002
EasyDL
3 wilderness 4:21 pm Oct 25, 2002
MarcoPolo
6 Xoc 3:19 pm Oct 25, 2002
rename asp files to html after the are handled by IIS
rename asp to html IIS
3 Clauss 2:48 pm Oct 25, 2002
Any experience with Webclipping robots?
5 QNetwork 9:13 pm Oct 24, 2002
mogimogi/1.0
goo robot?
3 PandaM 5:57 pm Oct 24, 2002
art site spider?
brain deadest spider I've ever seen
4 dcheney 3:25 am Oct 24, 2002
Does anyone know this crawler?
My site got hit by Almaden.IBM.com/cs/Crawler...
12 erikv 3:06 am Oct 24, 2002
Robots/Spiders visitors
Robots/Spiders visitors
2 Lalo 8:17 am Oct 23, 2002
Websquash
9 volatilegx 11:56 am Oct 22, 2002
PicaLoader
Looks like a good one to ban
1 carfac 4:49 pm Oct 20, 2002
ExactSeek
4 volatilegx 5:02 pm Oct 18, 2002
Identifying google bot
2 Joker 5:22 pm Oct 17, 2002
Cuam Ver0.050bx
2 Scooter24 4:49 pm Oct 17, 2002
Not a Spider...
Just want to know what the request is!
5 carfac 4:10 pm Oct 15, 2002
NutchCrawler
7 volatilegx 1:39 pm Oct 15, 2002
WorQmada/1.0
5 Scooter24 8:45 am Oct 15, 2002
PingALing
ANOTHER "is your website up" bot!
4 carfac 5:26 pm Oct 14, 2002
NG/1.0 update
Spider from France reads, ignores robots.txt
13 jdMorgan 5:08 pm Oct 14, 2002
webcrawler@nospam
4 Ove 3:44 pm Oct 14, 2002
GirafaBot
Who? What? Where?
8 frontpage 10:39 am Oct 14, 2002
URL Spider Pro
7 misosoph 10:58 pm Oct 13, 2002
Interesting visit by AskJeeves
Making a dead page cleanup pass?
5 jdMorgan 7:17 pm Oct 13, 2002
Why is important to know which spiders
are visiting?
7 dauction 3:42 am Oct 13, 2002
Mozilla/2.0
2 Scooter24 12:23 pm Oct 11, 2002