Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Using robot.txt to keep unwanted backlinking

Is this an effective means


Daily Sparring

4:57 pm on May 21, 2004 (gmt 0)

10+ Year Member

I have begun to hear recently that having backlinks from certain 'spam friendly' portals can be a very bad thing. Would using a robot.txt to keep these spiders out be effective, or would they just pull your data from your site anyways through other search engines?

I hope that makes sense, thanks in advance.


5:14 pm on May 21, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

I'm not quite sure what you're talking about but I'll try to answer what I guess you want to know. ;)

Having a robots.txt that prohibits real search engine spiders to go through your site mostly doesn't help to avoid backlinks. There are lots of bad bots out there masking as Mozilla or just not obeying the robots.txt. The only thing to handle this stuff is asking the site owners to remove the link. If they're stealing content, you can get the sites out by contacting the search engines.

I hope that met your question.


Featured Threads

Hot Threads This Week

Hot Threads This Month