Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Using robot.txt to keep unwanted backlinking

Is this an effective means

4:57 pm on May 21, 2004 (gmt 0)

Junior Member

10+ Year Member

joined:Dec 23, 2003
votes: 0

I have begun to hear recently that having backlinks from certain 'spam friendly' portals can be a very bad thing. Would using a robot.txt to keep these spiders out be effective, or would they just pull your data from your site anyways through other search engines?

I hope that makes sense, thanks in advance.

5:14 pm on May 21, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 5, 2002
votes: 0

I'm not quite sure what you're talking about but I'll try to answer what I guess you want to know. ;)

Having a robots.txt that prohibits real search engine spiders to go through your site mostly doesn't help to avoid backlinks. There are lots of bad bots out there masking as Mozilla or just not obeying the robots.txt. The only thing to handle this stuff is asking the site owners to remove the link. If they're stealing content, you can get the sites out by contacting the search engines.

I hope that met your question.