Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Pay-Per-Click engines. What value are they, really? |
6 | pendanticist | 6:02 am Feb 6, 2005 | ||||
| Leach of a Robot |
4 | merrick_lozano | 8:49 pm Feb 4, 2005 | ||||
| Blocking all but Mozzilla/G/Y/M Is this too simple? |
6 | dataguy | 5:45 pm Feb 2, 2005 | ||||
| Spider Issues: Relative Links Confusing Spiders A suggestion to use absolute links... |
4 | JAB_Creations | 3:10 am Feb 2, 2005 | ||||
| Clushbot/3.1-BinaryFury The newest Clush bot not checking robots.txt |
4 | pendanticist | 9:49 pm Feb 1, 2005 | ||||
| ConveraCrawler/0.4 any info? |
9 | jackson | 6:36 am Jan 31, 2005 | ||||
| innerprise Go Daddy |
2 | wilderness | 11:21 pm Jan 29, 2005 | ||||
| updated/0.1beta Checks robots.txt |
2 | pendanticist | 11:10 pm Jan 29, 2005 | ||||
| Vespa Crawler 3.9 |
3 | Odatpup | 2:53 pm Jan 27, 2005 | ||||
| Kitenga |
13 | wilderness | 5:57 am Jan 27, 2005 | ||||
| lwp-trivial/1.40 user agent which spider is it? |
3 | johnnie | 5:45 am Jan 26, 2005 | ||||
| Homerweb |
2 | wilderness | 2:29 pm Jan 25, 2005 | ||||
| what/who is "GT::WWW/1.023"? |
2 | Glamba | 9:27 pm Jan 24, 2005 | ||||
| ban ".NET CLR 1.1.4322" |
4 | vacuity | 1:48 pm Jan 24, 2005 | ||||
| how do you track? |
2 | sivko23 | 11:32 pm Jan 22, 2005 | ||||
| ZipppBot/0.11 Yet another directory |
2 | idoc | 1:55 pm Jan 21, 2005 | ||||
| GigabotSiteSearch/2.0 |
4 | volatilegx | 7:16 pm Jan 19, 2005 | ||||
| YottaCars_Bot no robots. |
8 | bull | 8:45 am Jan 18, 2005 | ||||
| become spider be aware of become.com |
5 | tomasz | 11:46 pm Jan 17, 2005 | ||||
| Are these spiders from Thailand? repreat visits - who are they |
8 | Gregse | 2:55 pm Jan 17, 2005 | ||||
| oegp v. 1.3.0 Whatis? |
3 | Yidaki | 12:16 am Jan 16, 2005 | ||||
| Trend Micro.......? Anybody heard of this one? |
2 | mcneely | 12:04 am Jan 16, 2005 | ||||
| Cyveillance back on new IP's Cyveillance, 38.118.25.56 - 38.118.25.63, 38.118.42.32 - 38.118.42.39 |
6 | gordongecko | 9:06 pm Jan 14, 2005 | ||||
| 8 | idoc | 1:23 am Jan 14, 2005 | |||||
| goforit.com |
7 | whitehat | 2:13 pm Jan 12, 2005 |
