Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| Starting to worry....IP picking up pages automatically when I upload 169.207.238.197 |
11 | kstprod | 7:43 pm Nov 26, 2002 | ||||
| Curiosity about Ask Jeeves How Ask Jeeves agent works |
4 | Tegame | 6:27 pm Nov 26, 2002 | ||||
| NaverRobot!= nabot? A simple question really... |
3 | Dreamquick | 8:14 am Nov 25, 2002 | ||||
| Extending to Qwest from Cyvelliance |
4 | wilderness | 8:42 pm Nov 24, 2002 | ||||
| 22000 Pages in 48 hours |
3 | cmoewes | 4:55 pm Nov 24, 2002 | ||||
| Colorado Internet Cooperative Association 199.45.128.0 - 199.45.255.255 |
5 | wilderness | 1:05 am Nov 24, 2002 | ||||
| Rogue bots or legitimate spiders? |
11 | dan_popescu | 1:02 am Nov 24, 2002 | ||||
| Oracle Ultra Search Seems that I'm just full of questions tonight... |
4 | Dreamquick | 12:21 pm Nov 23, 2002 | ||||
| Crawl 1,2,4,7 Have I been listed? |
5 | cyclic | 5:49 pm Nov 22, 2002 | ||||
| No, Spider! Stay! What does it mean when they grab your robots.txt page and then leave? |
4 | brina | 5:33 pm Nov 22, 2002 | ||||
| Who is this? Just grabbed my whole site twice |
9 | Powdork | 5:05 pm Nov 22, 2002 | ||||
| WebEco/1.0 Anyone know what it is? |
2 | Kerrin | 7:10 am Nov 22, 2002 | ||||
| phpgetter strange activity from 208.17.76.138 |
4 | PandaM | 3:12 am Nov 22, 2002 | ||||
| New one to me Quibot 3.0 |
3 | cmoewes | 8:59 pm Nov 21, 2002 | ||||
| NetNoseCrawler/v1.0 no robots.txt |
2 | Finder | 8:36 pm Nov 21, 2002 | ||||
| "HTML Text Download Class" Agent Does not respect robots.txt |
3 | carfac | 4:35 pm Nov 21, 2002 | ||||
| Sneaky Fast Spider |
4 | Josk | 4:33 pm Nov 21, 2002 | ||||
| Anyone know this bot? 216.55.138.108 |
8 | Weblamer | 10:10 pm Nov 20, 2002 | ||||
| AV Scooter 1.0 images |
16 | wilderness | 11:35 pm Nov 19, 2002 | ||||
| server1.business2www.com b2w/0.1 |
9 | Brett_Tabke | 10:10 am Nov 18, 2002 | ||||
| Scooter 3.2 IS Ruining My Marriage Someone tell me what is going on please |
9 | brina | 6:07 pm Nov 17, 2002 | ||||
| Does anyone know anything about these three user agents? Download Ninja 7.0, Iria/1.07a, and Pockey/4.10.2(Win32; GUI; ix86) |
3 | GaryK | 3:12 pm Nov 17, 2002 | ||||
| webcollage/1.87 no robots.txt |
7 | bull | 5:15 pm Nov 16, 2002 | ||||
| IBM Crawl_Application 198.81.209.19 |
2 | WebGuerrilla | 12:54 am Nov 16, 2002 | ||||
| Links 2.1pre3 legitimate browser |
2 | Finder | 9:41 pm Nov 15, 2002 |
