Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Netscape 5.0 is driving me nuts
Netscape 5.0 showing on logs
9 walkman 9:00 pm Sept 16, 2011
  Does everyone use bespoke scripts for spider identification
6 scooterdude 9:47 pm Sept 14, 2011
  Sitebuilder name in results-spammers thru a backdoor?
Sitebuilder in search results-block "backdoor"
5 KaraScene1 4:24 pm Sept 14, 2011
  Traffic from Malicious IPs and Adwords ads for my website
I get visitors from adwords when I never started a campaign
5 Makis77 8:45 pm Sept 13, 2011
  isc header collector
2 Pfui 9:30 pm Sept 9, 2011
  Iparadigms
8 keyplyr 7:01 pm Sept 3, 2011
  Why is this agent fetching my images?
7 aristotle 4:58 pm Sept 3, 2011
  PostPost
2 Pfui 1:52 pm Sept 3, 2011
  pflab
3 keyplyr 12:50 am Sept 3, 2011
  Netvigator
5 keyplyr 11:31 pm Sept 1, 2011
  ScooperBot
3 Pfui 2:46 pm Sept 1, 2011
  who is Covario and what do they want from me?
3 lucy24 7:27 pm Aug 31, 2011
  74.125.52.93 in my logs today
6 apauto 9:11 pm Aug 30, 2011
  AIM7 WinHTTP/1.0
2 Pfui 7:00 pm Aug 29, 2011
  YodaoBot
7 Pfui 1:16 am Aug 29, 2011
  BOTW Spider
Jurgle LLC in New York
2 grandma_genie 6:50 pm Aug 24, 2011
  jikespider
Chinese bot shape-shifts on the fly.
4 Pfui 7:29 pm Aug 23, 2011
  Google IPs but MSIE UA
Huh?
2 Pfui 4:47 pm Aug 22, 2011
  NerdByNature.Bot
13 keyplyr 10:18 pm Aug 21, 2011
  Level3 and more
5 wilderness 8:51 pm Aug 21, 2011
  AhrefsBot
née SiteBot
5 Pfui 4:57 pm Aug 21, 2011
  Butterfly
2 Pfui 7:30 pm Aug 20, 2011
  integrity/3
3 Pfui 4:30 pm Aug 18, 2011
  CydralSpider
2 Pfui 7:58 pm Aug 17, 2011
  Strange User Header
3 yaashul 4:30 pm Aug 15, 2011