Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| New empty site started being visited by spiders |
2 | sren | 3:21 pm Dec 27, 2005 | ||||
| OpenIntelligenceData/1.1 |
2 | bobothecat | 3:19 pm Dec 27, 2005 | ||||
| Getting slammed with phony robots Is it my site or niche? Are others seeing this? |
22 | larryhatch | 7:41 pm Dec 23, 2005 | ||||
| Wavefire |
2 | keyplyr | 5:35 pm Dec 21, 2005 | ||||
| What's wrong with this simple .htaccess file? Works for one phony spider, not another .. |
17 | larryhatch | 12:57 am Dec 19, 2005 | ||||
| AOL users running bots. any tips for halting them? |
3 | davelms | 7:34 am Dec 13, 2005 | ||||
| Y!j-bsc/1.0 Yahoo JP indexer? |
3 | keyplyr | 2:38 pm Dec 11, 2005 | ||||
| zerxspid.? |
3 | mcneely | 11:18 pm Dec 8, 2005 | ||||
| Microsoft? sandcrawler - compatibility testing |
4 | fiestagirl | 7:20 am Dec 5, 2005 | ||||
| Been seeing a lot of this fake Googlebot lately |
2 | volatilegx | 11:41 pm Dec 2, 2005 | ||||
| String followed by phx.gbl What is this? |
3 | abbeyvet | 9:55 pm Dec 2, 2005 | ||||
| Bot-watching new to this concept ... how best to do this? |
4 | jcmoon | 7:54 am Dec 2, 2005 | ||||
| Yahoo! Slurp China |
12 | MaxM | 3:51 am Dec 1, 2005 | ||||
| voyager/1.0 |
5 | bobothecat | 8:32 am Nov 23, 2005 | ||||
| OutfoxBot For internet experiments |
3 | GaryK | 11:28 pm Nov 22, 2005 | ||||
| InfoPath.1 |
8 | wilderness | 7:38 pm Nov 21, 2005 | ||||
| Is this a new search spider? |
8 | sculptex | 12:44 pm Nov 19, 2005 | ||||
| Microsoft Scheduled Cache Content Download Service/Fetch API Request |
6 | DrGUID | 6:56 am Nov 16, 2005 | ||||
| Trend Micro what do they want with our pdfs? |
3 | Mokita | 6:30 pm Nov 15, 2005 | ||||
| Orbiter |
9 | wilderness | 1:49 am Nov 14, 2005 | ||||
| Shim Crawler From University of Tokyo |
7 | GaryK | 2:03 pm Nov 13, 2005 | ||||
| RB2B-bot v0.1 |
4 | Mokita | 4:22 am Nov 13, 2005 | ||||
| Pingo? The pingo spider |
4 | johnwards | 12:36 am Nov 9, 2005 | ||||
| OutfoxBot/0.3 |
2 | bobothecat | 12:35 am Nov 9, 2005 | ||||
| CsCrawler University of Kassel, Germany |
2 | bull | 6:54 am Nov 8, 2005 |
