Welcome to WebmasterWorld Guest from 23.20.239.237

Forum Moderated by: Ocean10000 & incrediBILL & keyplyr

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
How can you identify spiders without log access
3 lazyz 8:02 pm Sep 7, 2002
Hits by Mozzilla/3.0- Not too polite
2 carfac 6:06 pm Sep 7, 2002
DTS Agent
Any new info on this UA?
3 SmallTime 9:38 pm Sep 6, 2002
What spider is this?
I found it in the kitchen....
4 Reeper 3:35 am Sep 6, 2002
Python-urllib/1.15
more RR
3 wilderness 11:34 am Sep 5, 2002
New Spider FS-WebC
3 skirril 10:16 am Sep 5, 2002
Who/what is: e-f-w-m-s:INET
1 onlineleben 5:39 pm Sep 4, 2002
proxy.google.com, UA: UP.Browser/3.1.04-SY03 UP.Link/5.0.2.3c (Google
11 martin 7:20 am Sep 3, 2002
I caught you.
Ok, you got me. So now what?
3 bobriggs 9:53 am Sep 2, 2002
empyreum.cz
empyreum spider search engine czech
4 weesnich 3:34 am Sep 1, 2002
LOA v.1.4)
1 wilderness 5:53 pm Aug 31, 2002
Cyveillance behaving badly
guess I'll write them a nice letter
10 idiotgirl 10:44 pm Aug 30, 2002
Scooter-3.2.PDF
3 wilderness 11:20 am Aug 30, 2002
A real winner
No doubt about this one's UA!
6 jdMorgan 10:06 pm Aug 29, 2002
Propfind?
Useragent: Microsoft-WebDAV-MiniRedir/5.1.2600
3 frontpage 4:14 pm Aug 29, 2002
64.156.198.74
19 wilderness 3:41 pm Aug 28, 2002
Epsilon Softworks' MailMunky
on my endangered species list
12 idiotgirl 5:45 am Aug 27, 2002
NetX HPe Thin Client
6 wilderness 3:23 am Aug 26, 2002
User Agent = btiv6download
4 Woz 3:07 am Aug 26, 2002
UncoverMaroco bot
anybody seen him?
1 korkus2000 9:46 pm Aug 23, 2002
Mewsoft Search Engine
Spider does not check robots.txt
1 jdMorgan 1:41 am Aug 23, 2002
gsa-crawler
Google for hire?
5 wilderness 10:49 am Aug 22, 2002
Up to date search engine spider IP lists.
anyone?
4 caine 11:35 pm Aug 21, 2002
spider4.picsearch
? new ?
2 Birdman 4:35 am Aug 21, 2002
Microsoft Data Access Internet Publishing Provider
What is it?
2 Pushycat 3:26 am Aug 21, 2002

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members