Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| LargeSmall Crawler |
3 | Pfui | 12:20 am Sept 23, 2009 | ||||
| Yahoo |
16 | wilderness | 7:11 pm Sept 21, 2009 | ||||
| Domain Tools Now Using GurujiBot IP is from Name Intelligence |
4 | incrediBILL | 7:27 pm Sept 20, 2009 | ||||
| MSN's cloak-crawling again: Twitter / Tweets Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0) |
3 | Pfui | 6:32 pm Sept 20, 2009 | ||||
| Fake Googlebot requests on the rise Botnet, competitive intelligence, or semi-legit? |
6 | jdMorgan | 6:00 pm Sept 20, 2009 | ||||
| Gist Server |
2 | Pfui | 4:17 pm Sept 20, 2009 | ||||
| msnbot/2 snapshots? |
9 | keyplyr | 6:44 am Sept 20, 2009 | ||||
| Wget/1.10.2 |
3 | outland88 | 9:24 pm Sept 19, 2009 | ||||
| Cuil in stealth mode? |
4 | keyplyr | 10:05 am Sept 19, 2009 | ||||
| Mozilla/4.0 MSN strikes (out) again. |
11 | Pfui | 9:30 am Sept 19, 2009 | ||||
| Cat & Dog Chinese UA |
3 | dstiles | 1:41 am Sept 19, 2009 | ||||
| Zeta Interactive? Keep getting hit by bots from 64.40.121.nnn |
7 | madmatt69 | 5:53 pm Sept 18, 2009 | ||||
| Twisted PageGetter |
2 | Pfui | 3:58 pm Sept 17, 2009 | ||||
| findfiles.net/0.96 |
2 | keyplyr | 1:41 pm Sept 17, 2009 | ||||
| ScrapBook Site scraper bot; also discloses user's local file info |
27 | Pfui | 2:21 pm Sept 16, 2009 | ||||
| AutoPager Egads |
13 | incrediBILL | 10:03 am Sept 16, 2009 | ||||
| solelynetwork |
2 | dstiles | 9:44 pm Sept 14, 2009 | ||||
| www.travel-agent-finder.co.uk |
2 | GaryK | 8:42 am Sept 14, 2009 | ||||
| Firefox Ubuntu (jaunty) crawler or screen shots? From 09/08/24 to 09/13/24 and still going |
5 | incrediBILL | 9:33 pm Sept 13, 2009 | ||||
| open proxies major providers |
3 | wilderness | 10:47 pm Sept 10, 2009 | ||||
| Earthlink mobile |
3 | wilderness | 9:15 pm Sept 10, 2009 | ||||
| MJ12bot When you're a hammer, everything looks like a nail[2] ( 1 2 ) |
38 | Pfui | 7:18 pm Sept 7, 2009 | ||||
| searchevabot |
2 | Pfui | 8:04 am Sept 7, 2009 | ||||
| MSN fakes referrers [5] ( 1 2 3 4 5 ) |
131 | SEOPTI | 6:47 am Sept 7, 2009 | ||||
| Known Spider Information and References |
12 | incrediBILL | 9:15 am Sept 5, 2009 |
