Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Anyone know what AuditBot/0.1 is Being hit everyday with this |
10 | dramstore | 11:36 am Oct 4, 2005 | ||||
| LocalcomBot/1.2 Claims to index local websites. |
10 | GaryK | 4:07 am Oct 3, 2005 | ||||
| ShunixBot New French search engine |
4 | GaryK | 7:20 pm Oct 2, 2005 | ||||
| Metaspinner |
2 | wilderness | 6:33 am Oct 2, 2005 | ||||
| PrivacyFinder Cache Bot |
3 | keyplyr | 12:26 am Sept 29, 2005 | ||||
| Yahoo requesting GET vars that don't exist It appears that slurp is auto-incrementing a get variable |
3 | stargeek | 12:36 am Sept 27, 2005 | ||||
| Lycos |
4 | volatilegx | 8:46 pm Sept 26, 2005 | ||||
| Longest User Agent Ever? Pure Spam |
6 | volatilegx | 7:18 pm Sept 25, 2005 | ||||
| iCCrawler - any info? claims to come from iccenter.net - purpose? |
2 | thetrasher | 7:42 pm Sept 24, 2005 | ||||
| zzrobots.txt .yeah human error my [bleep]! |
6 | JAB_Creations | 8:38 pm Sept 21, 2005 | ||||
| Anyone read Finnish? Virgo/0.7 |
9 | keyplyr | 8:03 pm Sept 21, 2005 | ||||
| PreviewGenerator.org |
2 | MLHmptn | 2:01 pm Sept 20, 2005 | ||||
| Project Kolinka Forum Search Another bot that ignores robots.txt |
8 | Moparx | 4:18 am Sept 20, 2005 | ||||
| SBIder/0.7 What do you think about this bot? |
8 | GaryK | 2:49 am Sept 20, 2005 | ||||
| Clushbot Again This time in the Laomedon flavor |
3 | GaryK | 12:47 am Sept 20, 2005 | ||||
| Reputable Spider lists? Trying to find a reputable spider list... what do you recommend? |
3 | profitpuppy | 12:41 am Sept 20, 2005 | ||||
| Omnipelagos.com RSS Spider 0.3 Claims to find shortest path between any two things. |
2 | GaryK | 12:39 am Sept 20, 2005 | ||||
| Feedfetcher-Google; (+http://www.google.com/feedfetcher.html) Just noticed this one today... |
2 | BillyS | 11:32 pm Sept 19, 2005 | ||||
| Dynamic+ Does anyone know what this crawler/company does? |
3 | GaryK | 4:24 pm Sept 19, 2005 | ||||
| MPFv7.0.0.153 To script or not to script? |
2 | JAB_Creations | 11:51 pm Sept 18, 2005 | ||||
| D:\\@\\5\\5.exe |
2 | JAB_Creations | 1:06 am Sept 15, 2005 | ||||
| Lycos_Spider_(modspider) |
2 | WarmGlow | 8:43 pm Sept 14, 2005 | ||||
| CFNetwork/1.1 useragent |
2 | JAB_Creations | 4:07 pm Sept 14, 2005 | ||||
| Weirdly formated UA requested robots.txt |
2 | Dijkgraaf | 4:06 am Sept 14, 2005 | ||||
| Vision Research Lab |
8 | wilderness | 5:46 pm Sept 8, 2005 |
