Forum Moderated by: open

Crawler, Spider, and User Agent ID


Forum to identify search engine spiders and user agents

 
Thread SubjectMessagesStarted byLast Message
  Python-urllib
from google?
7 silverbytes 1:48 pm Sept 26, 2003
  "Image Collector V1.0"
2 seindal 1:42 pm Sept 26, 2003
  Colocation
66.114.67.120
8 wilderness 9:02 pm Sept 25, 2003
  cj.com spider?
5 jp_19 4:05 pm Sept 25, 2003
  Been hit hard lately!
3 creative_craig 12:28 pm Sept 25, 2003
  205.223.216.6
6 coyote 2:20 am Sept 25, 2003
  Host name: j3174.inktomisearch.com
All Over My forum's for Hours
3 epic1231 3:32 pm Sept 24, 2003
  FAST-WebCrawler/3.8/Fresh
Is this new?
2 mipapage 11:33 pm Sept 23, 2003
  Overture-WebCrawler
6 Tor 6:29 pm Sept 23, 2003
  Internetseer
How do I kill this parasite?
14 epanastatis 11:50 pm Sept 22, 2003
  Can anyone imagine/knows what Googlebot's Interface is like?
Just wondering
4 sidyadav 8:56 pm Sept 22, 2003
  203.24.143.197
bot used to steal?
4 coyote 6:47 pm Sept 21, 2003
  Intelliseek spybot / snoopbot / copyrightbot
any other IP ranges for this pest
2 nativenewyorker 6:23 pm Sept 21, 2003
  Name Protect bot a big pest
How to ban it?
13 mayor 6:05 pm Sept 21, 2003
  Program Shareware 1.0.3
What is this?
9 andrewrab 1:15 am Sept 20, 2003
  RPT-HTTPClient/0.3-3
New IBM bot or just odd browser?
2 claus 11:35 pm Sept 19, 2003
  New User_Agent "PHP/4.1.1"
Who knows about this UA?
3 Wizcrafts 4:27 pm Sept 19, 2003
  New ZyBorg UA strings - again
Two of them - this time featuring "Beta"
2 claus 3:56 pm Sept 19, 2003
  27 Googlebot's on my site
...all at the same time
5 esllou 6:29 pm Sept 18, 2003
  MSIE Crawler, visits daily, follows overture listing
A nasty little 'spider' that is costing me money
4 werty 4:12 pm Sept 18, 2003
  EgotoBot/4.8
spiders dmoz listing, "caching" pages
2 stevenha 11:40 am Sept 16, 2003
  Yahoo is up to something
207.126.224.174
4 fiestagirl 6:20 am Sept 16, 2003
  baiduspider
new IP for baiduspider, fwiw
5 kpaul 10:24 pm Sept 15, 2003
  37.104 and 39.104
what are these, please?
8 beezee 9:04 pm Sept 15, 2003
  Beginner needs spider monitoring software
4 nicheman 4:20 pm Sept 15, 2003