Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| LWP::Simple heads up |
9 | wilderness | 7:23 pm July 21, 2003 | ||||
| New UA: CopyHunter/1.54 Has anybody else seen it and know its purpose? |
3 | Wizcrafts | 10:42 am July 21, 2003 | ||||
| SetEnvIf ... keep_out (Apache directive) |
3 | Made_In_Sheffield | 10:18 pm July 19, 2003 | ||||
| Cyveillance IP Ranges What are the IP Ranges for Cyveillance |
9 | kewlbeezer | 7:30 pm July 18, 2003 | ||||
| Willow Internet Crawler twotrees.com |
2 | bull | 9:03 pm July 17, 2003 | ||||
| Analyze your log: how to exclude yourself? How to avoid being counted? Your own visits to your website... |
6 | silverbytes | 9:33 pm July 16, 2003 | ||||
| Whois this guy?- 12.119.251.194 GOT index 670 times so far today |
16 | Powdork | 12:43 am July 16, 2003 | ||||
| A Simple Beginning with htaccess |
3 | wilderness | 11:58 pm July 15, 2003 | ||||
| Mediapartners-Google/2.1 But no adsense/adwords or similar |
3 | pixel_juice | 9:34 pm July 15, 2003 | ||||
| Chinese bot from 61.144.218.161 Anyone've heard of this Chinese bot? |
2 | aleksl | 1:11 pm July 15, 2003 | ||||
| guestbook spammers webbots that are sniffing around guestbook pages |
23 | Jaf | 12:21 pm July 15, 2003 | ||||
| Some Google hints 216.33.229.163 and Python again |
8 | pontifex | 8:56 pm July 14, 2003 | ||||
| 65.110.131.177 one hungry bot |
2 | coyote | 11:48 am July 14, 2003 | ||||
| Look out GoogleBot! Look out MSNBot! Here comes GalaxyBot! New bot from galaxy.com |
7 | Made_In_Sheffield | 11:11 am July 14, 2003 | ||||
| WebmasterWorldWebBot on 193.95.242.56 |
7 | jimmykav | 11:35 am July 12, 2003 | ||||
| 209.189.108.196 oingo.com Requested disallowed files |
5 | upside | 10:28 pm July 11, 2003 | ||||
| 210.196.42.200 Asia Pacific Network Information Centre |
4 | photoman | 6:20 pm July 11, 2003 | ||||
| inktomi slurp robot. si1006.inktomisearch.com |
2 | jackobar | 3:07 pm July 10, 2003 | ||||
| "semanticdiscovery" revisited... sleddogcafe was (sort of) right all along! :) |
4 | balam | 1:16 pm July 10, 2003 | ||||
| gemima/1.0 new ua to me |
3 | zipzip | 12:21 pm July 10, 2003 | ||||
| anyone recongnise 24.63.10.222? |
2 | BCNman | 9:02 am July 10, 2003 | ||||
| Szukacz |
2 | wilderness | 11:20 pm July 9, 2003 | ||||
| Maybe a spider? "Crawl_Application" Has not been brought up for a few months. |
2 | werty | 5:48 pm July 9, 2003 | ||||
| GornKer Crawler - Anyone know what it does? It did 1600 pageviews on my site in a week! |
3 | ir8Rtist | 5:11 pm July 9, 2003 | ||||
| FYI: New Googlebot IPs |
5 | Marcia | 1:11 pm July 9, 2003 |
