Welcome to WebmasterWorld Guest from 54.224.96.57

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

How to noindex specific directories with X-Robots Tag?

.htaccess and duplicate content

     
7:05 pm on Jun 12, 2014 (gmt 0)

New User

joined:June 12, 2014
posts:2
votes: 0


Hey everyone,

My client has a joomla-based website, and his products create duplication issues. Ex:

example.com/vanities/product/show/51-wall-mounted-vanity-black-box-black-sink.html
example.com/hikashop-menu-for-module-186/product/show/51-wall-mounted-vanity-black-box-black-sink/category_pathway-66.html
example.com/aquamobilia/product/show/51-wall-mounted-vanity-black-box-black-sink.html

(I've altered the links not too look like I'm spamming)

Basically, we'd like anything in /aquamobilia and /hikashop-menu-for-module-186 to be noindex nofollow, without the pain of doing it for each individual page.

Is it possible to do it with X-Robots Tag in .htaccess? I'll admit I'm very bad at programming and with Apache.

Thanks a lot for your help,

Charles

[edited by: phranque at 9:00 pm (utc) on Jun 12, 2014]
[edit reason] exemplify domain [/edit]

9:34 am on June 14, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13064
votes: 307


It can be done, but it's a little tricky because you can't have <Directory> sections in htaccess, so you need to put a separate little htaccess file in each directory that you want to protect.

It's a single line:
Header set X-Robots-Tag "noindex"

If you wanted to, you could add a <FilesMatch> constraining it to specific filenames or extensions-- but I don't suppose your page files are surrounded by non-page files in the same directories that you do want indexed.

But wait! This can only be done if you're dealing with real, physical directories. You said joomla. Do those / slashes in the URL represent physical directories?

:: wandering off to see if there's a way to set headers within mod_rewrite ::
12:19 pm on June 16, 2014 (gmt 0)

New User

joined:June 12, 2014
posts:2
votes: 0


Wow, I had been looking for such an answer for hours; thanks a lot Lucy!