Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| TeragramCrawler how to block this search engine crawler |
3 | mswiryn | 11:06 pm Mar 31, 2006 | ||||
| FreshNotes crawler |
4 | keyplyr | 11:07 pm Mar 29, 2006 | ||||
| Can I ban Mediapartners-Google/2.1? |
4 | joker197cinque | 6:31 pm Mar 28, 2006 | ||||
| differences between real and fake googlebots how to recognize real googlebot |
8 | hurgada | 3:52 am Mar 28, 2006 | ||||
| Current Googlebot IP's For geo-targetting between UK and US sites |
4 | TinkyWinky | 3:48 am Mar 28, 2006 | ||||
| csci_b659/0.13 |
3 | wilderness | 11:03 pm Mar 22, 2006 | ||||
| Msn? |
2 | wilderness | 8:35 pm Mar 22, 2006 | ||||
| more Hurricane |
2 | wilderness | 8:16 pm Mar 22, 2006 | ||||
| Yet another Yahoo spider UA Appears to be for .de search |
3 | jdMorgan | 10:51 pm Mar 21, 2006 | ||||
| New Googlebot Nokia6820/2.0 |
8 | Mokita | 5:01 am Mar 20, 2006 | ||||
| FAST-WebCrawler/2.2.5 - Lycos/Alltheweb/Fast Something very fishy about this one |
13 | jdMorgan | 11:44 pm Mar 19, 2006 | ||||
| Adsense Bot Identifing Adsense bot |
4 | vortech | 11:07 pm Mar 19, 2006 | ||||
| Java/1.5.0_06 Spider Sighting, and Questions |
7 | avendus | 12:23 am Mar 19, 2006 | ||||
| OrangeSpider |
3 | Pfui | 5:39 am Mar 18, 2006 | ||||
| vermut vermut.aol.com, coming from india |
6 | thetrasher | 5:51 am Mar 15, 2006 | ||||
| Verizon Spider? |
3 | trinorthlighting | 9:13 pm Mar 14, 2006 | ||||
| Octora Beta |
4 | Pfui | 9:01 pm Mar 14, 2006 | ||||
| hl_ftien_spider Bot from China |
3 | lammert | 1:42 pm Mar 12, 2006 | ||||
| sproose/0.1-alpha |
6 | bobothecat | 11:14 pm Mar 11, 2006 | ||||
| When Google's Not Looking Am I A Bad Person For Asking This? |
4 | inuwolf | 5:57 pm Mar 9, 2006 | ||||
| Unnamed spider on 67.15.184.* Very rapid crawling, hundreds of requests / minute |
6 | andye | 3:24 pm Mar 9, 2006 | ||||
| Keegeebot |
4 | volatilegx | 5:01 pm Mar 8, 2006 | ||||
| New Spider IP Address This IP doesn't show up on any listings I've seen |
2 | ThirdRock | 2:11 pm Mar 7, 2006 | ||||
| Rss_reader |
2 | bobothecat | 11:23 pm Mar 6, 2006 | ||||
| ucbot |
2 | bobothecat | 10:54 pm Mar 2, 2006 |
