Welcome to WebmasterWorld Guest from 54.227.101.214

Forum Moderated by: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification


PreModerated forum to id search engine spiders

 
Thread SubjectMessagesStarted byLast Message
How to report a potential Bot abuse?
Would love to have one in my draft folder just in case ;)
2 hanuman 4:15 pm Jun 14, 2002
Reading robots.txt or root directory, then leaving
3 Bluestreak 4:02 pm Jun 14, 2002
New company Name
Any Ideas for our new company name?
2 canucks1980 8:07 pm Jun 13, 2002
New Ink Spider
5 fiestagirl 4:09 pm Jun 12, 2002
Group of new Inktomi IPs
1 volatilegx 3:42 pm Jun 12, 2002
Dbrowse 1.4b, RSurf, PEval
what are they?
8 Jaf 3:38 pm Jun 12, 2002
Proper syntax for banning bad bots in htaccess
using mod_rewrite and htaccess commands correctly for banning bad bots
8 Bluestreak 2:42 pm Jun 12, 2002
Mysterious User Agent is random string , AQJPTSIP, e.g.
Anybody know what kind of masking agent is being used and why?
8 nonprof_webguy 2:04 pm Jun 12, 2002
List of user agents
2 dang4 8:57 am Jun 11, 2002
puf/0.91beta6a
Has this robot harmed anyone?
4 misosoph 6:17 am Jun 11, 2002
Spider Identificatioon script
How many visits from googlebot ? Referals ? etc..
2 Thierry_Zoller 9:53 am Jun 10, 2002
Inktomi Random Agent header ?
MonkeysNuts/5.01 [Its the dogs)
3 Thierry_Zoller 9:46 am Jun 10, 2002
Yet more Alavaista User-Agents
When will ever stop making new ones?
2 Josk 9:45 am Jun 10, 2002
Fast WebCrawler 3.6
Is this new?
4 coconutz 6:27 pm Jun 9, 2002
66.150.40.220
Can anyone identify it?
5 Birdman 11:25 am Jun 9, 2002
FirstGov.gov Search - POC:firstgov.webmasters@gsa.gov
Is this for real?
5 Key_Master 5:15 am Jun 9, 2002
Brainysearch
64.57.64.90
1 volatilegx 11:45 pm Jun 7, 2002
Snoopy_v0.1
4 korkus2000 3:34 pm Jun 5, 2002
BlitzBOT@tricus.net ?
3 bull 3:54 pm Jun 4, 2002
65.24.37.57 - toto
toto? - no referrer, no ua
6 misosoph 4:32 am Jun 4, 2002
Unwanted Spiders - how to limit/kill access
simple ways to code, spiders list etc.
3 lak12 7:56 pm Jun 3, 2002
scooter/1.0 (netscan.fr)
Is this Alta Vista?
3 volatilegx 5:51 pm Jun 3, 2002
Monitoring bot activity with a script?
8 Bluestreak 1:00 am Jun 3, 2002
Charter
My apologies
1 wilderness 9:54 pm Jun 2, 2002
autoemailspider
email harvester or poorly named spider?
3 zem 12:48 am Jun 2, 2002

Featured Threads

Hot Threads This Week

Hot Threads This Month