Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Some unique visitors |
6 | wilderness | 5:00 pm May 14, 2003 | ||||
| Googleperson? 216.239.37.5 |
3 | coconutz | 1:25 pm May 14, 2003 | ||||
| can anyone identify this ASPseek/1.2.10 |
2 | indiandomain | 6:31 am May 14, 2003 | ||||
| AsiaNetBot anyone knows this one? |
3 | bull | 11:40 pm May 13, 2003 | ||||
| SurveyBot/2.2 |
6 | elektra | 6:56 pm May 13, 2003 | ||||
| ask.24x.info No Crawler Name 62.75.136.40 |
2 | jim_w | 12:52 pm May 13, 2003 | ||||
| Anyone heard of this one tide72.microsoft.com |
2 | totter | 7:53 am May 13, 2003 | ||||
| blocking Road Runner |
10 | cayleyv | 9:37 am May 10, 2003 | ||||
| Enticing a spider to crawl the site Robots.txt or no Robots.txt |
6 | SinclairUser | 1:01 am May 10, 2003 | ||||
| 216-239-45-4.google.com Not in adwords, so why'd it come by? |
4 | mipapage | 8:03 am May 9, 2003 | ||||
| 128.255.244.90 "Generic" Seen this one? |
2 | Busynut | 3:51 am May 8, 2003 | ||||
| misbehaved bots how to trap misbehaved bots? |
10 | indiandomain | 6:50 pm May 7, 2003 | ||||
| It's Official! MicrosoftPrototypeCrawler is legitimate! |
17 | pendanticist | 10:45 pm May 6, 2003 | ||||
| Heavy spidering from ICANN 206.196.54.148 |
6 | contracosta | 9:19 pm May 6, 2003 | ||||
| Been hit hard this month! All major SE bots visited this month |
4 | creative_craig | 12:47 pm May 6, 2003 | ||||
| 65.223.250.253 Sugar Frosted Flakes? |
22 | wilderness | 3:35 pm May 3, 2003 | ||||
| New spider or what? semanticdiscovery.com is the site but.... |
13 | coconubuck | 3:30 pm May 3, 2003 | ||||
| 24.77.72.205 What is this thing! |
5 | JVB_Mktg | 4:40 am May 3, 2003 | ||||
| Comprehensive List of Spiders good and bad? |
2 | TheWebographer | 1:07 pm May 2, 2003 | ||||
| VoilaBot is a Spambot Though it supposed to run for a search engine |
13 | WebJoe | 8:38 am May 2, 2003 | ||||
| dloader(NaverRobot)/1.0 switching IP Numbers very, very quickly. |
4 | pendanticist | 8:09 am May 2, 2003 | ||||
| Do you know when? First apperence of BackRub |
5 | Honor_to_BackRub | 4:15 am May 2, 2003 | ||||
| www.indiscipline.net/robots.txt odd referrer |
2 | Andrue | 12:00 am May 1, 2003 | ||||
| New Crawler - Comodo Seems to be an SSL Crawler |
7 | philipzae | 6:38 pm Apr 30, 2003 | ||||
| KaHT |
2 | jimmykav | 6:33 pm Apr 30, 2003 |
