Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Omnipelagos.com RSS Spider 0.3 Claims to find shortest path between any two things. |
2 | GaryK | 12:39 am Sept 20, 2005 | ||||
| Feedfetcher-Google; (+http://www.google.com/feedfetcher.html) Just noticed this one today... |
2 | BillyS | 11:32 pm Sept 19, 2005 | ||||
| Dynamic+ Does anyone know what this crawler/company does? |
3 | GaryK | 4:24 pm Sept 19, 2005 | ||||
| MPFv7.0.0.153 To script or not to script? |
2 | JAB_Creations | 11:51 pm Sept 18, 2005 | ||||
| D:\\@\\5\\5.exe |
2 | JAB_Creations | 1:06 am Sept 15, 2005 | ||||
| Lycos_Spider_(modspider) |
2 | WarmGlow | 8:43 pm Sept 14, 2005 | ||||
| CFNetwork/1.1 useragent |
2 | JAB_Creations | 4:07 pm Sept 14, 2005 | ||||
| Weirdly formated UA requested robots.txt |
2 | Dijkgraaf | 4:06 am Sept 14, 2005 | ||||
| Vision Research Lab |
8 | wilderness | 5:46 pm Sept 8, 2005 | ||||
| Help me with my .htaccess file please |
2 | a_angova | 3:07 am Sept 8, 2005 | ||||
| 200 (return error code 0) - What does it means? |
2 | Shaundm | 1:56 pm Sept 7, 2005 | ||||
| envolk[ITS]spider |
4 | wilderness | 1:51 pm Sept 7, 2005 | ||||
| JRSocketEngine? |
4 | teasers | 3:33 am Sept 7, 2005 | ||||
| SlurpConfirm404 Is this really Slurp? |
4 | fish_eye | 6:34 pm Sept 6, 2005 | ||||
| Strange UA |
4 | wilderness | 1:31 am Sept 5, 2005 | ||||
| Mozilla/4.0 (compatible; prgrabber 1.0) Is this a Google PageRank checker? |
2 | GaryK | 6:17 pm Sept 4, 2005 | ||||
| Amazon Crawling aws-alexa libwww-perl/5.65 |
9 | The_Contractor | 5:37 pm Sept 4, 2005 | ||||
| msnbot/1.0-MM (+http://search.msn.com/msnbot.htm) Comming from 65.54.164.### |
4 | caspita | 2:39 pm Sept 2, 2005 | ||||
| Yahoo Spider consuming large amounts of bandwidth |
3 | servoweb | 3:09 pm Sept 1, 2005 | ||||
| Checkbot and LWP University uses to check links |
2 | guitaristinus | 3:05 pm Aug 31, 2005 | ||||
| InfociousBot (+http://corp.infocious.com/tech_crawler.php) and another bad bot |
3 | DoppyNL | 1:32 pm Aug 30, 2005 | ||||
| Ocp |
4 | wilderness | 2:08 am Aug 30, 2005 | ||||
| Yahoo spidering from new Class C subnet |
5 | volatilegx | 1:44 pm Aug 29, 2005 | ||||
| Unknown spiders visiting |
3 | Dave_A | 10:49 pm Aug 28, 2005 | ||||
| double slashes |
2 | wilderness | 7:01 pm Aug 28, 2005 |
