Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| New bot Java/1.5.0_06 grabs all pages grabbed all pages from 2 different domains |
5 | privacyman | 7:04 am Feb 9, 2006 | ||||
| GigaBot - which engine sent this crawling? |
21 | Event_King | 7:30 pm Feb 8, 2006 | ||||
| Burf Ignore Robot.txt |
9 | frontpage | 1:09 pm Feb 6, 2006 | ||||
| Revisiting MSNbot and Google Translator |
7 | volatilegx | 10:46 pm Feb 4, 2006 | ||||
| snprtz |
8 | wilderness | 6:07 am Jan 29, 2006 | ||||
| Hotbot-Lycos or Slurp? Smarter Stats getting confused? |
7 | roodle | 11:12 am Jan 27, 2006 | ||||
| How Differentiate Users from Bots? Offering different access |
7 | grahamberends | 11:50 pm Jan 26, 2006 | ||||
| Inktomi - 68.142.211.201 |
2 | classifieds | 4:57 pm Jan 24, 2006 | ||||
| Anupa Browser or spider? |
2 | GaryK | 4:54 pm Jan 23, 2006 | ||||
| enoola crawler |
2 | keyplyr | 10:17 pm Jan 21, 2006 | ||||
| List of Spiders to Block blocking spiders for web analytics reporting |
8 | chicagoSEO | 7:49 pm Jan 20, 2006 | ||||
| Vision Research Lab image spider no robots |
2 | bull | 4:31 pm Jan 20, 2006 | ||||
| Which Google Bot UA is correct? I'm seeing two variations on the 2.1 bot |
4 | GaryK | 3:49 pm Jan 20, 2006 | ||||
| Exabot/2.0 |
5 | bumpaw | 10:08 am Jan 20, 2006 | ||||
| How do you make sure a new, genuine, spider is not blocked? |
3 | inbound | 9:53 am Jan 12, 2006 | ||||
| TeragramCrawler |
4 | bobothecat | 9:49 am Jan 11, 2006 | ||||
| plinki/0.1 you got plinked! (thats a good thing..) |
3 | DanA | 8:34 pm Jan 10, 2006 | ||||
| Banning APNIC What would be the reprocutions |
5 | twist | 7:50 am Jan 7, 2006 | ||||
| Vortex/2.2 |
4 | bobothecat | 8:10 pm Jan 5, 2006 | ||||
| LapozzBot? |
4 | keyplyr | 8:44 am Dec 28, 2005 | ||||
| New empty site started being visited by spiders |
2 | sren | 3:21 pm Dec 27, 2005 | ||||
| OpenIntelligenceData/1.1 |
2 | bobothecat | 3:19 pm Dec 27, 2005 | ||||
| Getting slammed with phony robots Is it my site or niche? Are others seeing this? |
22 | larryhatch | 7:41 pm Dec 23, 2005 | ||||
| Wavefire |
2 | keyplyr | 5:35 pm Dec 21, 2005 | ||||
| What's wrong with this simple .htaccess file? Works for one phony spider, not another .. |
17 | larryhatch | 12:57 am Dec 19, 2005 |
