Forum Moderators: open
#Hit:----#reqs:---#pages:---browser
5:------1342:------342:-----InternetSeer.com
8:-------226:-------226:----lwp-trivial
---------226:-------226:----lwp-trivial/1
11:------178:-------154:----libwww-perl
--:------178:-------154:----libwww-perl/5
12:------185:-------145:----Marvin v0.3
13:-----2079:--------84:----MOTHERF***** (compatible; 6PAK; Windows)
14:-----4428:--------84:----Konqueror
:-------4316:--------84:----Konqueror/3
15:-------65:--------65:----Scrubby
--:-------65:--------65:----Scrubby/2
16:-------72:--------62:----Microsoft Data Access Internet Publishing Provider Protocol Discovery
17:-------55:--------50:----AgentName
--:-------55:--------50:----AgentName/0
18:-------41:--------41:----SurveyBot
--:-------41:--------41:----SurveyBot/2
19:-------42:--------37:----ExactSeek Crawler
20:-------40:--------32:----Web Downloader
[edited by: littleman at 4:47 am (utc) on June 20, 2003]
[edit reason] language [/edit]
See the Close to Perfect Ban List.
A very long and worthwhile thread.
[webmasterworld.com...]
...someone please tell weather i should block them.
InternetSeer offers a link verification service. They mostly just make HEAD requests (zero bytes transferred) so I just ignore them. I don't think it is worth the two minutes of my time that it will take to add the robot name to my robots.txt file and add the User Agent to my .htaccess banned robots list.
They have an incompetent marketing department that sent me a UCE message soliciting their service. They found a broken link on one of my pages. The test was done from a page that they had indexed four months prior to the date of the test. Dumb! Dumb! Dumb! I do not do business with companies that send UCE.
