Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Univ of Toyko Just a heads up |
2 | wilderness | 10:53 pm July 2, 2003 | ||||
| Firefly |
2 | WebJoe | 11:59 am July 2, 2003 | ||||
| New Excite bot? Similar IP but no ID |
3 | too_much_information | 12:09 pm July 1, 2003 | ||||
| grub and ZyBorg what's going on? |
8 | claus | 8:17 am July 1, 2003 | ||||
| 212.190.252.64.snet.net - any ideas? it's a spider for sure, but what is it? |
3 | claus | 7:16 am July 1, 2003 | ||||
| 203.219.236.102 Who is this? |
14 | Stefan | 12:04 am July 1, 2003 | ||||
| Cxhttp New UA to me |
3 | MarieC | 11:28 pm June 30, 2003 | ||||
| AltaVista Robots What are they called |
2 | lloyd | 7:29 pm June 30, 2003 | ||||
| Which spiders do you want crawling your site? |
9 | ktd121 | 5:32 pm June 30, 2003 | ||||
| Two Bots, 1/2 a world apart- Same Requests This is weird! |
3 | carfac | 4:20 am June 30, 2003 | ||||
| WebGather 3.0 ignores robot.txt |
2 | frontpage | 6:05 am June 29, 2003 | ||||
| 217.155.46.193 heads-up |
4 | wilderness | 10:35 pm June 27, 2003 | ||||
| 168.71.5.202 and a loopy Roadrunner |
2 | coyote | 2:05 pm June 27, 2003 | ||||
| Mozilla/8.0 stupid spammer and other bots |
4 | coyote | 11:16 pm June 26, 2003 | ||||
| SiteWinder heads-up |
3 | wilderness | 11:01 pm June 26, 2003 | ||||
| MicrosoftPrototypeCrawler - Is it new? It came to my site. |
9 | IITian | 4:46 pm June 26, 2003 | ||||
| MarkWatch/1.0 snoopbot another IP to add to the block list |
2 | nativenewyorker | 11:13 pm June 25, 2003 | ||||
| Univ. of Aizu Web Crawler New *evil* spider |
4 | dcheney | 9:09 pm June 25, 2003 | ||||
| MnogoSearch just a heads up |
3 | wilderness | 7:07 pm June 24, 2003 | ||||
| 194.203.75.29 heads up |
4 | wilderness | 11:06 pm June 23, 2003 | ||||
| Custo just noticed this |
2 | carfac | 4:43 pm June 23, 2003 | ||||
| Could this be a bot? only hits homepage |
7 | too_much_information | 3:21 pm June 23, 2003 | ||||
| wysiwyg is this a spider unknown referre |
19 | nilloc | 2:55 pm June 23, 2003 | ||||
| What's This? IE+5.5+Compatible+Browser |
2 | xcandyman | 11:39 am June 23, 2003 | ||||
| UA for GetPageSize? |
5 | keyplyr | 11:05 pm June 22, 2003 |
