Forum Moderators: open

Message Too Old, No Replies

Teradex Mapper violating robots.txt

in fact, they never bothered to check

         

Finder

10:28 pm on Dec 27, 2002 (gmt 0)

10+ Year Member


Teradex Mapper has never looked at my robots file and only ever tried to take an old DMOZ listed page. About a week ago I added that directory to my robots file and Teradex just tried a peek at it.

I emailed the contact address and received an auto-reply that wanted me to click on a link and manually type in a password of sorts in order to verify my email address. Almost as if the contact addy is white-listed.

Needless to say, it's far more satisfying to add them to my httpd.conf.

UA: (Teradex Mapper; mapper@teradex.com; http://www.teradex.com)

carfac

6:08 am on Dec 29, 2002 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Finder:

Banned that one a while ago.... just for that reason. Looking at the site, seems to be a DMOZ-based directory. Not too impressibve...

dave