Forum Moderators: phranque

Message Too Old, No Replies

Allowing only selected sites to link

         

webdesigner43

7:45 am on Aug 6, 2009 (gmt 0)

10+ Year Member



What I would like to do is stop the automatic bots taking my website and placing a link and/ or content on their own website, and adding information which may or may not be accurate or wished for.

One way would be to ban all websites that I do not agree with how they treat the website. However this is a lot of work for me, and I am also finding myself playing catch up. And no offense to the people from these websites, but I would really like someone to ask permission first.

What I would like to do is create a mod rewrite rule that would block all external referrers/ domains. It has been awhile since I have been up to speed with this. But essentially allow people visiting directly or through a search engine such as Google or Yahoo to view the website. Additionally if some one wants to link to the website, and they have been given permission.

I could then write a webpage with something like "How to link to this website" and have for example a contact form which people could ask for permission.

In other words 'opt-in' instead of 'opt-out'.

I looked at the:
[webmasterworld.com...]
and
[webmasterworld.com...]

topics, and I have experimented with this before. However I want to be sure I don't disallow the search engine bots that I do want. So for example blocking all, but allowing 'google.com' might not be the best way to go. As IP adresses changes this could cause problems.

So pseudo code would be:

Deny all
if google
or yahoo
or bing
or site allowed to link
or regular visitor
Allow

If behavior appears to be bot not on white list, but for example masquarading as a normal visitor.
Deny

Possibly:
Redirect all denies to page explain terms of service, and that permission is needed for the link to be placed.

Would this also work for Iframes?

Nothing against some sites, but my biggest gripe is people using my website (to make money), and sometimes displaying incorrect information about my website, because they are using a robot, and did not take enough time to figure out why, how, or what. It doesn't kill you to ask permission so I can decide whether or not I want to participate. Mind you people adding a manual link, I find generally spend more time doing research, and actually providing a quality link.

Thank you for any assistance.