Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Use Spamhaus PBL to Decode Mixed Use Space?
10 incrediBILL 10:28 pm Dec 15, 2011
  Blekko Scoutjet new IP range
7 dstiles 10:58 pm Dec 14, 2011
  CloudFlare and spammers?
How can CloudFlare be used by spammers?
8 Hope_Fowl 7:43 pm Dec 13, 2011
  opera/9.80
3 topr8 8:20 am Dec 13, 2011
  AdBot
4 keyplyr 9:52 pm Dec 12, 2011
  GoogleProducer
New UA from Bare G IPs
16 Pfui 10:05 pm Dec 11, 2011
  TwitterBot
3 incrediBILL 1:25 pm Dec 11, 2011
  orange you sorry you asked?
6 lucy24 8:06 am Dec 10, 2011
  Netriplex
5 keyplyr 1:33 am Dec 10, 2011
  Nigma.ru
2 Pfui 10:02 pm Dec 3, 2011
  Kindle
7 Pfui 7:38 pm Dec 3, 2011
  Opera Night at Amazon Web Services
2 incrediBILL 6:17 pm Dec 3, 2011
  ContextAd Bot
2 keyplyr 7:15 pm Nov 30, 2011
  spxyric1.bankofamerica.com
3 keyplyr 10:20 pm Nov 28, 2011
  Lycosa - anyone know what this is?
15 motorhaven 1:59 pm Nov 27, 2011
  80legs IPs
to post or not to post
8 incrediBILL 7:07 am Nov 27, 2011
  Mozenda
2 incrediBILL 9:02 pm Nov 26, 2011
  sem rush bot and Amazon IPs
Is this an Amazon bot or a bot using Amazon?
3 grandma_genie 7:51 pm Nov 26, 2011
  SearchBot
3 keyplyr 12:47 am Nov 24, 2011
  Host-Spy
2 Pfui 8:18 am Nov 22, 2011
  Pixray? Ixnay .
2 Pfui 1:58 am Nov 20, 2011
  Hostpro Ltd, Ukraine
2 keyplyr 8:46 pm Nov 19, 2011
  Baiduspider - does it obey robots.txt ?
[2] ( 1 2 )
34 1script 10:37 pm Nov 18, 2011
  Microsoft bot 157 ranges updated
DNS for IP ranges 157.55 and 157.56 updated
21 dstiles 6:03 pm Nov 18, 2011
  about.com
4 keyplyr 8:50 pm Nov 16, 2011