Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Great lookup tool Great program |
6 | tcmalaska | 7:14 pm Mar 1, 2001 | ||||
| Concentric Network Corporation |
4 | msgraph | 2:15 pm Mar 1, 2001 | ||||
| SnoopRob Where does SnoopRob come from? |
3 | pikaia | 7:41 am Mar 1, 2001 | ||||
| User-Agent: MyLycos Please, this is all over my stuff |
16 | han solo | 7:25 pm Feb 28, 2001 | ||||
| Ip:199.245.188.74 UA: libwww-perl/5.50 |
4 | roscoepico | 6:43 pm Feb 28, 2001 | ||||
| who is this? |
5 | hohmaster | 5:30 pm Feb 28, 2001 | ||||
| Newbie - Do Spiders eat JPG files? I could not identify this IP |
8 | Macguru | 12:34 pm Feb 28, 2001 | ||||
| Who/What is this? |
2 | jk321 | 8:09 am Feb 27, 2001 | ||||
| Crawler6.bos2.fast-search.net FAST / Alltheweb crawler? |
5 | Rumbas | 5:28 am Feb 25, 2001 | ||||
| Make them stop! any luck? |
8 | luckynh | 1:46 pm Feb 21, 2001 | ||||
| 211.117.61.68 |
4 | Froggyman | 8:45 pm Feb 20, 2001 | ||||
| Sidewinder What is it doing? |
3 | Hope | 6:50 pm Feb 20, 2001 | ||||
| JennyBot/0.1? 64.97.116.103; JennyBot/0.1 |
5 | skirril | 2:27 am Feb 20, 2001 | ||||
| Netcraft.. ariston.netcraft.com ; Mozilla 4.0 Compat.. |
3 | skirril | 10:23 pm Feb 17, 2001 | ||||
| Where can I optain Spider List? Where can I optain Spider List? |
9 | clienttell | 12:51 pm Feb 16, 2001 | ||||
| Why are non search engines robots hitting our site? |
2 | Acternaweb | 4:27 pm Feb 15, 2001 | ||||
| Spoofed Mozilla/4.72 Acts like a robot, doesn't send a Host header |
2 | Kimihia | 3:09 am Feb 14, 2001 | ||||
| Robots Searching For .... ? interesting question |
4 | zon2k | 9:34 am Feb 13, 2001 | ||||
| Wonder what AV are testing (is MC the new, hip AV) |
5 | vis | 9:22 am Feb 13, 2001 | ||||
| Zyborg? |
3 | jlara | 12:06 am Feb 13, 2001 | ||||
| who is InterNAP... and why do they keep hitting / |
6 | mivox | 7:36 pm Feb 12, 2001 | ||||
| German spiders up to date list of german spiders |
5 | jan_zandbergen | 11:18 am Feb 10, 2001 | ||||
| New Ink Spiders |
9 | BoneHeadicus | 12:36 am Feb 9, 2001 | ||||
| Spider foot print What does this term mean |
9 | startup | 10:16 am Feb 5, 2001 | ||||
| Fast Spider? Looks like Fast...smells like Fast... |
7 | BoneHeadicus | 5:53 am Feb 4, 2001 |