Welcome to WebmasterWorld Guest from 54.166.203.76

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
rename asp files to html after the are handled by IIS
rename asp to html IIS
3 Clauss 2:48 pm Oct 25, 2002
Any experience with Webclipping robots?
5 QNetwork 9:13 pm Oct 24, 2002
mogimogi/1.0
goo robot?
3 PandaM 5:57 pm Oct 24, 2002
art site spider?
brain deadest spider I've ever seen
4 dcheney 3:25 am Oct 24, 2002
Does anyone know this crawler?
My site got hit by Almaden.IBM.com/cs/Crawler...
12 erikv 3:06 am Oct 24, 2002
Robots/Spiders visitors
Robots/Spiders visitors
2 Lalo 8:17 am Oct 23, 2002
Websquash
9 volatilegx 11:56 am Oct 22, 2002
PicaLoader
Looks like a good one to ban
1 carfac 4:49 pm Oct 20, 2002
ExactSeek
4 volatilegx 5:02 pm Oct 18, 2002
Identifying google bot
2 Joker 5:22 pm Oct 17, 2002
Cuam Ver0.050bx
2 Scooter24 4:49 pm Oct 17, 2002
Not a Spider...
Just want to know what the request is!
5 carfac 4:10 pm Oct 15, 2002
NutchCrawler
7 volatilegx 1:39 pm Oct 15, 2002
WorQmada/1.0
5 Scooter24 8:45 am Oct 15, 2002
PingALing
ANOTHER "is your website up" bot!
4 carfac 5:26 pm Oct 14, 2002
NG/1.0 update
Spider from France reads, ignores robots.txt
13 jdMorgan 5:08 pm Oct 14, 2002
webcrawler@nospam
4 Ove 3:44 pm Oct 14, 2002
GirafaBot
Who? What? Where?
8 frontpage 10:39 am Oct 14, 2002
URL Spider Pro
7 misosoph 10:58 pm Oct 13, 2002
Interesting visit by AskJeeves
Making a dead page cleanup pass?
5 jdMorgan 7:17 pm Oct 13, 2002
Why is important to know which spiders
are visiting?
7 dauction 3:42 am Oct 13, 2002
Mozilla/2.0
2 Scooter24 12:23 pm Oct 11, 2002
Lexibot
Allow or ban
4 mack 11:38 pm Oct 10, 2002
rabaz
Seems well behaved
2 carfac 5:19 pm Oct 10, 2002
ua: ht
3 carfac 4:50 pm Oct 10, 2002

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members