Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Would this work? Robots exclusion or considered cloaking?



9:36 am on Jul 8, 2005 (gmt 0)

10+ Year Member


we have a travel site which has about a thousand pages, and has been online in the search engines for about 5 years. it uses a CMS, one that outpts in html.

we have never done any real seo, apart from changing a few tags around since last year. however it ranks ok for the keywords we have chosen.

we are going to do a total re-design. the new design proposed has a lot of rich media content, is still using the cont tent management system, and will be very graphic oriented.

if we made a duplicate version for accessibility reasons , in plain html, exactly like the bbc have done . (u can see this under 'text only' version on the bbc main page - bbc.co.uk/home/today/textonly.shtml) and then put a robot exclusion tags on every page of the rich media site. could this work?

so in effect we have a index splash page. 2 entry points on that page. main visitors will go to the colourful graphic pages. that link blocks the search engines. they follow the links to the text only pages. previous link popularity still going to main index page, + new links deeplinking to text only pages.

is this considered a kind of cloaking? or is it just an alternative way of doing things. we would have to have the accessability part anyway for legal reasons as it is a brand site. this way would just be taking advantage of seo at the same time.

we would also need to have the robot exclusion tags because of the duplicate content. this way there would only be 1 set of copy indexed and it would just be the easy to spider text stuff.

any help, info or ideas would be appreciated thanks.

Abdelrhman Fahmy

1:38 am on Jul 10, 2005 (gmt 0)

10+ Year Member

i don't think that is a duplicate by any way as its an added value to your site visitors ,
also you could put all of the plain text Content in a separate folder and block the access to the whole folder by Robots.txt


3:41 am on Jul 10, 2005 (gmt 0)

10+ Year Member

I'd like to hear everyone's input on this one too.

Logically, this should work as you are not attempting to manipulate the SEs with the graphical pages and if blocked, they should not appear to be "duplicates" of the text pages.

However, many pages I have blocked with NOINDEX, NOFOLLOW turn up as URL only listings. In a couple of cases, where they were, in fact, near duplicates of an indexed page, the near duplicate indexed page has been dropped to URL only as well. Not the plan at all! Would like to know why.

Where I have put near duplicate pages in a separate directory (eg landing pages for AdWords), the directory blocked entirely by robots.txt, it seems to have been 99% effective: no sign of these in Google (except where a stray link may have crept in) and no related duplicate penalties.


7:56 am on Jul 14, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

dvldvl3, first of all Welcome to the robots.txt Forum at WebmasterWorld!

I'd definetly try to avoid having both pages indexed as this could cause problems with the search engines. I'd put, as already suggested, both versions in separate folders and disallow one of them via the robots.txt-Standard. This makes handling very easy as you don't have to adjust the robots.txt file every time you add/change/remove a file that's in the root directory.


5:11 am on Jul 17, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member

another way to accomplish this is with the same page but different .css files.
Definitely allow the text-based low-graphics page to be spidered.

Featured Threads

Hot Threads This Week

Hot Threads This Month