Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
Is this really a PlayStation 3
How can you tell if the browser is really a PlayStation 3?
12 Ocean10000 3:25 am Apr 7, 2008
Quick primer on identifying bot activity.
And a how to guide to slow and stop scraping
21 Ocean10000 12:48 pm Apr 4, 2008
Bot or malware?
I'm seeing strange 'msoffice' access to my system
4 Megaclinium 4:48 am Apr 4, 2008
Please welcome our new Moderator incrediBILL
29 engine 4:41 am Apr 4, 2008
Bravo MSN
2 wilderness 9:57 pm Apr 2, 2008
Yahoo image grabber
2 newbie6 8:08 pm Apr 2, 2008
identifying user views resulting from spiders
8 newbie6 2:28 pm Apr 2, 2008
Snoopy v1.2.3 Source Identified
PHP Class
3 incrediBILL 5:46 pm Mar 28, 2008
CCBot
another Nutch Variant
7 Ocean10000 1:39 pm Mar 28, 2008
Sap
6 wilderness 11:48 pm Mar 27, 2008
User Agent: "WordPress/2.1.1" coming from Russia and UK
Is this an RSS aggragator, or a scraper bot?
6 Wizcrafts 2:32 am Mar 27, 2008
Readrun
tag crawl showing up lately annoyance
4 mrjones 9:52 am Mar 26, 2008
Slashers
wondering about site slashers
3 Megaclinium 6:22 am Mar 26, 2008
what is this bot?
72.244.103.zz
3 Megaclinium 11:42 pm Mar 25, 2008
ConveraCrawler
Is ConveraCrawler a malicious robot?
17 mattie 1:05 pm Mar 25, 2008
Boitho ignoring robots.txt
7 Megaclinium 8:17 pm Mar 24, 2008
Strange UA
4 wilderness 1:14 am Mar 21, 2008
Pete-Spider Light
2 keyplyr 11:24 pm Mar 20, 2008
TekSavvy
8 wilderness 12:20 pm Mar 20, 2008
gnu-classpath
Cute
3 wilderness 9:27 pm Mar 19, 2008
zermelo
3 zerillos 8:34 pm Mar 19, 2008
odd access log entry
access log get form entry
2 esme 10:03 pm Mar 18, 2008
anyone know this UA?
all the spaces are +'s :?
11 wkitty42 4:07 pm Mar 18, 2008
agodar
3 Mokita 10:28 pm Mar 17, 2008
User Agent "Firefox"
8 marodhum 10:17 pm Mar 17, 2008