Forum Moderated by: open
Forum to identify search engine spiders and user agents
| Thread Subject | Messages | Started by | Last Message | ||||
|---|---|---|---|---|---|---|---|
| New Yahoo crawler? Strange crawler from yahoo |
5 | Strange | 5:54 pm Mar 16, 2005 | ||||
| Googlebot? |
6 | LeoXIV | 3:00 am Mar 16, 2005 | ||||
| Huge visit from 161.76.99.242 ip. Is their anything wrong? |
3 | nex99 | 5:08 pm Mar 15, 2005 | ||||
| Robots exclusion How to exclude selectively robots? |
3 | specter | 5:00 pm Mar 15, 2005 | ||||
| Anyone know if this robot is good or bad? |
4 | candlelady | 2:06 pm Mar 15, 2005 | ||||
| NWSpider 0.9 No robots.txt request. |
2 | pendanticist | 10:30 pm Mar 14, 2005 | ||||
| CostaCider - New Search pretty lame... ...when it throws you a MS-DOS application to dl. Huh? |
3 | pendanticist | 3:43 pm Mar 14, 2005 | ||||
| Spider? sasch1031302.phx.gbl |
3 | palmpal | 3:38 pm Mar 14, 2005 | ||||
| JemmaTheTourist ActiveTourist checks robots.txt |
3 | pendanticist | 8:34 am Mar 13, 2005 | ||||
| msnbot coming through Google IP address Now that's weird. |
5 | volatilegx | 6:12 pm Mar 11, 2005 | ||||
| BecomeBot/1.23 - New Version Seems to find my site thru those who link to me? |
6 | pendanticist | 5:18 pm Mar 11, 2005 | ||||
| How to check if an IP is a valid robot or search engine bot? |
4 | georgec | 8:16 am Mar 10, 2005 | ||||
| National Institute of Standards and Technology |
3 | wilderness | 5:09 pm Mar 9, 2005 | ||||
| Could you help me identify these robots? |
18 | skuba | 9:28 pm Mar 7, 2005 | ||||
| UnChaos Checks robots.txt |
2 | pendanticist | 5:07 pm Mar 7, 2005 | ||||
| Google mobile phone? |
2 | 993ti | 3:57 pm Mar 4, 2005 | ||||
| Ban IP or ban UA? |
6 | skuba | 1:13 am Mar 4, 2005 | ||||
| 200 connects from spiders? anyone seeing high connects from spiders |
3 | theceo | 12:04 am Mar 4, 2005 | ||||
| Strange Googlebot Anyone seen this little guy? |
10 | edit_g | 11:56 pm Mar 3, 2005 | ||||
| Is inktomi slurp still alive and surfing? |
3 | GreenLeaf | 11:32 pm Mar 3, 2005 | ||||
| Hit by Java/1.4.x_xx crawlers |
5 | moltar | 2:46 am Mar 2, 2005 | ||||
| tScholarsBot "tScholarsBot [beta] is now up and ready..." and checks robots.txt. |
3 | pendanticist | 4:31 am Mar 1, 2005 | ||||
| newsg8/0.1 Never heard of them |
2 | idoc | 1:54 pm Feb 26, 2005 | ||||
| Faxobot/1.0 Figured out what it is! |
10 | djzooky | 5:40 am Feb 25, 2005 | ||||
| KnowItAll Univ. Washington |
6 | bull | 4:12 pm Feb 21, 2005 |
