Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Help me with my .htaccess file please |
2 | a_angova | 3:07 am Sept 8, 2005 | ||||
| 200 (return error code 0) - What does it means? |
2 | Shaundm | 1:56 pm Sept 7, 2005 | ||||
| envolk[ITS]spider |
4 | wilderness | 1:51 pm Sept 7, 2005 | ||||
| JRSocketEngine? |
4 | teasers | 3:33 am Sept 7, 2005 | ||||
| SlurpConfirm404 Is this really Slurp? |
4 | fish_eye | 6:34 pm Sept 6, 2005 | ||||
| Strange UA |
4 | wilderness | 1:31 am Sept 5, 2005 | ||||
| Mozilla/4.0 (compatible; prgrabber 1.0) Is this a Google PageRank checker? |
2 | GaryK | 6:17 pm Sept 4, 2005 | ||||
| Amazon Crawling aws-alexa libwww-perl/5.65 |
9 | The_Contractor | 5:37 pm Sept 4, 2005 | ||||
| msnbot/1.0-MM (+http://search.msn.com/msnbot.htm) Comming from 65.54.164.### |
4 | caspita | 2:39 pm Sept 2, 2005 | ||||
| Yahoo Spider consuming large amounts of bandwidth |
3 | servoweb | 3:09 pm Sept 1, 2005 | ||||
| Checkbot and LWP University uses to check links |
2 | guitaristinus | 3:05 pm Aug 31, 2005 | ||||
| InfociousBot (+http://corp.infocious.com/tech_crawler.php) and another bad bot |
3 | DoppyNL | 1:32 pm Aug 30, 2005 | ||||
| Ocp |
4 | wilderness | 2:08 am Aug 30, 2005 | ||||
| Yahoo spidering from new Class C subnet |
5 | volatilegx | 1:44 pm Aug 29, 2005 | ||||
| Unknown spiders visiting |
3 | Dave_A | 10:49 pm Aug 28, 2005 | ||||
| double slashes |
2 | wilderness | 7:01 pm Aug 28, 2005 | ||||
| Can you help me with this IP is it spider |
2 | tomasz | 1:34 am Aug 28, 2005 | ||||
| Jakarta Commons-HttpClient/3.0-rc1 Anyone seeing this? |
18 | The_Contractor | 5:41 pm Aug 27, 2005 | ||||
| DynaWeb? Chinese proxy service |
4 | keyplyr | 5:15 pm Aug 27, 2005 | ||||
| SquigglebotBot/1.0 Is this related to Relevare? |
3 | GaryK | 5:06 pm Aug 27, 2005 | ||||
| Jayde |
3 | wilderness | 11:06 am Aug 26, 2005 | ||||
| YahooSeeker-Testing/v3.9 testing what ... I wonder? |
6 | bobothecat | 1:24 pm Aug 25, 2005 | ||||
| Oracle Ultra Search no robots |
3 | bull | 2:43 pm Aug 23, 2005 | ||||
| Becomebot |
4 | reaper | 3:27 pm Aug 22, 2005 | ||||
| DTAAgent |
3 | bumpaw | 2:26 am Aug 22, 2005 |
