homepage Welcome to WebmasterWorld Guest from 54.211.219.178
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Visit PubCon.com
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
How to noindex specific directories with X-Robots Tag?
.htaccess and duplicate content
Cheebee




msg:4679450
 7:05 pm on Jun 12, 2014 (gmt 0)

Hey everyone,

My client has a joomla-based website, and his products create duplication issues. Ex:

example.com/vanities/product/show/51-wall-mounted-vanity-black-box-black-sink.html
example.com/hikashop-menu-for-module-186/product/show/51-wall-mounted-vanity-black-box-black-sink/category_pathway-66.html
example.com/aquamobilia/product/show/51-wall-mounted-vanity-black-box-black-sink.html

(I've altered the links not too look like I'm spamming)

Basically, we'd like anything in /aquamobilia and /hikashop-menu-for-module-186 to be noindex nofollow, without the pain of doing it for each individual page.

Is it possible to do it with X-Robots Tag in .htaccess? I'll admit I'm very bad at programming and with Apache.

Thanks a lot for your help,

Charles

[edited by: phranque at 9:00 pm (utc) on Jun 12, 2014]
[edit reason] exemplify domain [/edit]

 

lucy24




msg:4679841
 9:34 am on Jun 14, 2014 (gmt 0)

It can be done, but it's a little tricky because you can't have <Directory> sections in htaccess, so you need to put a separate little htaccess file in each directory that you want to protect.

It's a single line:
Header set X-Robots-Tag "noindex"
If you wanted to, you could add a <FilesMatch> constraining it to specific filenames or extensions-- but I don't suppose your page files are surrounded by non-page files in the same directories that you do want indexed.

But wait! This can only be done if you're dealing with real, physical directories. You said joomla. Do those / slashes in the URL represent physical directories?

:: wandering off to see if there's a way to set headers within mod_rewrite ::

Cheebee




msg:4680252
 12:19 pm on Jun 16, 2014 (gmt 0)

Wow, I had been looking for such an answer for hours; thanks a lot Lucy!

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved