Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Clean sites use cloaking?

I have to cloak certain links



4:11 pm on Nov 1, 2005 (gmt 0)

10+ Year Member

I run quite an large forum (200,000 posts). All my links are permalinks and properly formatted without any querystrings etc - every level can be navigated.

I have rss and Print versions of threads. I cloak those links to remove them from the SERPS (robots.txt doesn't use wildcards / regex!) I don't want to have 3 versions of one thread (original / print / rss) in any search engines. C# has a problem adding rel nofollow to hyperlinks too.

I cloak drop-downs and other pointless parts (like login links / join now / addpost links) - otherwise i get hundreds of thousands of those pages in google etc..

99% of visual content remains identical, only some links and dropdowns etc are missing in the SE cached version.

Is this wrong? I can't get the BOTS to stop pulling these pages and wasting my B/W in any other way.


2:05 pm on Nov 3, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

Is this wrong? I can't get the BOTS to stop pulling these pages and wasting my B/W in any other way.

From an ethical/moral point of view it certainly isn't wrong. What you do with your website is your business. This cloaking would be done to benefit the users and the search engines, eliminating possible multiple listings of the same page.

However, there are other ways to accomplish your goal without cloaking, such as Mod_Rewrite. Through the use of Mod_Rewrite, you could redirect spiders from your RSS/print versions to the original version with either temporary or permanent redirect.

Alternatively, you could insert NOINDEX meta tags into the RSS/print versions.


3:59 pm on Nov 4, 2005 (gmt 0)

10+ Year Member

Well I used to use NOINDEX or Content NONE etc, but the index shows thousands of empty urls (which i didn't like!)

I used various redirect techniques initially, but that leaves 200,000+ spider links that 301 to individual pages.

Now i've 410d (404 leaves pages in the index) the pages I don't want, and removed the redirects. I've cloaked the links so no more are added.


Featured Threads

Hot Threads This Week

Hot Threads This Month