Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| NameProtect.com's NPBot bot hides in plain sight via IP 12.175.0.44 Rarely asks for, never heeds robots.txt |
2 | Pfui | 4:53 pm Apr 26, 2006 | ||||
| crawl-66-249-66-39.googlebot.com never seen this one in the host logs |
3 | hu12 | 4:04 pm Apr 26, 2006 | ||||
| Patwebbot |
3 | bobothecat | 12:23 am Apr 26, 2006 | ||||
| zedzo |
2 | wilderness | 7:49 am Apr 25, 2006 | ||||
| Bacon Media Bot Bacon Media Bot |
3 | ByteEnable | 3:57 am Apr 25, 2006 | ||||
| Was my site crawled? |
2 | TuanLa1972 | 2:21 am Apr 25, 2006 | ||||
| lanshanbot/1.0 |
2 | bobothecat | 2:33 am Apr 21, 2006 | ||||
| msnbot visiting in pairs? |
2 | bobothecat | 11:41 pm Apr 20, 2006 | ||||
| csci_b659/0.13 |
3 | bull | 7:07 pm Apr 20, 2006 | ||||
| noyona_0_1 |
3 | Staffa | 3:34 pm Apr 20, 2006 | ||||
| WebCopier v4.3? What is it? Found this in my logs |
4 | Essex_boy | 7:59 pm Apr 19, 2006 | ||||
| 66.151.103.144 & 66.151.103.181 |
2 | hfactor | 11:14 am Apr 19, 2006 | ||||
| Yahoo! Mindset Didn't ask for robots.txt |
8 | Pfui | 11:07 pm Apr 15, 2006 | ||||
| LG/U8120/v1.0 ignoring robots, nocache tags |
6 | keyplyr | 2:43 pm Apr 14, 2006 | ||||
| Strange Yahoo Bot I think |
3 | innocbystr | 7:31 pm Apr 13, 2006 | ||||
| What is the name the Yahoo image bot? or is it a seperate bot? |
5 | Iczer | 2:15 pm Apr 12, 2006 | ||||
| Interesting from Yahoo! |
3 | volatilegx | 12:29 pm Apr 12, 2006 | ||||
| 64.233.173.67 Google spider |
14 | Key_Master | 3:03 am Apr 11, 2006 | ||||
| schibstedsokbot |
4 | bobothecat | 9:23 pm Apr 10, 2006 | ||||
| Google Wireless Transcoder |
3 | Umbra | 6:22 pm Apr 7, 2006 | ||||
| InfoPath.1 How to block |
3 | JadedJane | 4:28 pm Apr 7, 2006 | ||||
| Strange spidering something is trying to find non existing files... |
5 | netchicken1 | 12:15 pm Apr 6, 2006 | ||||
| Comrite/0.7.1 comrot robot identification query |
5 | fusion5 | 11:33 pm Apr 3, 2006 | ||||
| rel="nofollow" confusion don't follow |
7 | Drumat5280 | 8:49 pm Apr 3, 2006 | ||||
| Spider Design walking the tightrope |
5 | BerndH | 7:27 pm Apr 3, 2006 |
