Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Is this new? NetNose-Crawler 2.0 |
2 | jim_w | 1:11 pm May 30, 2003 | ||||
| Need help identifying a spider what spider is this? |
3 | tim3562 | 11:06 pm May 29, 2003 | ||||
| ETS v5.1 what is it? |
7 | bobmark | 4:33 pm May 29, 2003 | ||||
| More Cogeco Outdated thread |
8 | wilderness | 4:24 am May 29, 2003 | ||||
| Microsofts .NET hitting my site like crazy Need help to stop it |
10 | DrOliver | 2:12 pm May 28, 2003 | ||||
| Halo - the magical bot |
2 | jimmykav | 9:38 am May 28, 2003 | ||||
| Here is a reason NOT to block blank U-As It just might be this website checking for valid code |
9 | Wizcrafts | 6:21 pm May 27, 2003 | ||||
| need help with 403 custom page not working in .htaccess I included an ErrorDocument 403 custom page but I get error message |
9 | Wizcrafts | 5:34 pm May 26, 2003 | ||||
| Need to block a specific spider Help with robots txt |
7 | coosblues | 9:38 pm May 25, 2003 | ||||
| Visits from 'mercury.hiddenservers.com' Is this a Google bot? |
5 | salson | 1:37 pm May 25, 2003 | ||||
| kulokobot backweave |
3 | DavidT | 8:07 pm May 24, 2003 | ||||
| Why is deepbot using freshbot ip? |
2 | requiem | 7:44 pm May 24, 2003 | ||||
| wizard.dataforce.net kore wa, nani? (= what's this?) |
4 | berli | 7:43 pm May 23, 2003 | ||||
| freshbot instead ob deepbot 216.x.x.x 64.x.x.x classification still true? |
3 | plasma | 5:23 pm May 23, 2003 | ||||
| Is this Googlebot or an imposter? New type of entry in my log files |
4 | birdstuff | 9:03 pm May 22, 2003 | ||||
| Googlebot? 216.239.39.x |
10 | Stefan | 8:37 pm May 22, 2003 | ||||
| 207.42.75.170 |
8 | DavidT | 4:44 am May 22, 2003 | ||||
| How do I set up a spider trap? badwidth theft |
3 | Digger | 11:55 pm May 21, 2003 | ||||
| 218.63.199.179 The China Farm Bank QUJING Branch |
3 | carfac | 3:45 pm May 21, 2003 | ||||
| ElectricSurfMaster::CHTMLTagGenSearcher Webwasher.com GmbH |
3 | jim_w | 2:42 pm May 21, 2003 | ||||
| MixCat Out Crawling or Log Spamming? |
3 | EliteWeb | 2:02 pm May 21, 2003 | ||||
| How to exclude linko/0.1 libwww-perl/5.66 |
5 | ovtech | 4:45 am May 21, 2003 | ||||
| 212.150.53.131 |
4 | ircgeeks | 6:01 am May 20, 2003 | ||||
| Qango.com Web Directory http://www.qango.com/ |
2 | DavidT | 6:47 pm May 19, 2003 | ||||
| 139.81.162.6 |
2 | wilderness | 6:42 pm May 19, 2003 |
