Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| spider or scraper? based on IP, how can you get the REAL STORY? |
12 | arnarn | 4:58 pm May 8, 2006 | ||||
| Mediabot coming through Yahoo! Proxy |
9 | volatilegx | 11:59 pm May 7, 2006 | ||||
| Microsoft-WebDAV - What was this trying to do? |
7 | JAB_Creations | 7:33 am May 7, 2006 | ||||
| Gt::www/1.026 a PHP/Perl class? |
2 | hakre | 4:33 pm May 5, 2006 | ||||
| KakleBot |
2 | bobothecat | 7:24 pm May 4, 2006 | ||||
| Explore XY |
11 | wilderness | 4:16 pm May 4, 2006 | ||||
| OutfoxBot /0.1? Anyone know who what where? |
2 | fusion5 | 3:41 pm May 4, 2006 | ||||
| Tarantula/0.1 |
3 | bumpaw | 8:33 pm May 1, 2006 | ||||
| TAIS Pinger (A SQL thing? An MSN thing?) Looked for (unlinked) referring site's files on my site. Loose w/ privacy. |
3 | Pfui | 6:23 pm Apr 26, 2006 | ||||
| NameProtect.com's NPBot bot hides in plain sight via IP 12.175.0.44 Rarely asks for, never heeds robots.txt |
2 | Pfui | 4:53 pm Apr 26, 2006 | ||||
| crawl-66-249-66-39.googlebot.com never seen this one in the host logs |
3 | hu12 | 4:04 pm Apr 26, 2006 | ||||
| Patwebbot |
3 | bobothecat | 12:23 am Apr 26, 2006 | ||||
| zedzo |
2 | wilderness | 7:49 am Apr 25, 2006 | ||||
| Bacon Media Bot Bacon Media Bot |
3 | ByteEnable | 3:57 am Apr 25, 2006 | ||||
| Was my site crawled? |
2 | TuanLa1972 | 2:21 am Apr 25, 2006 | ||||
| lanshanbot/1.0 |
2 | bobothecat | 2:33 am Apr 21, 2006 | ||||
| msnbot visiting in pairs? |
2 | bobothecat | 11:41 pm Apr 20, 2006 | ||||
| csci_b659/0.13 |
3 | bull | 7:07 pm Apr 20, 2006 | ||||
| noyona_0_1 |
3 | Staffa | 3:34 pm Apr 20, 2006 | ||||
| WebCopier v4.3? What is it? Found this in my logs |
4 | Essex_boy | 7:59 pm Apr 19, 2006 | ||||
| 66.151.103.144 & 66.151.103.181 |
2 | hfactor | 11:14 am Apr 19, 2006 | ||||
| Yahoo! Mindset Didn't ask for robots.txt |
8 | Pfui | 11:07 pm Apr 15, 2006 | ||||
| LG/U8120/v1.0 ignoring robots, nocache tags |
6 | keyplyr | 2:43 pm Apr 14, 2006 | ||||
| Strange Yahoo Bot I think |
3 | innocbystr | 7:31 pm Apr 13, 2006 | ||||
| What is the name the Yahoo image bot? or is it a seperate bot? |
5 | Iczer | 2:15 pm Apr 12, 2006 |
