Welcome to WebmasterWorld Guest from 23.22.46.195

Forum Moderators: goodroi

How do I disallow bots from my Wordpress year Archive Folders

I guess this is a common question but I can't find an answer

   
1:35 am on Apr 16, 2007 (gmt 0)

10+ Year Member



How do I disallow bots from spidering my archive folders in Wordpress?

Here is how my URL's are structured:

http://www.example.com/blog/index.php/2007/04/

I tried testing this in Webmaster Central, but it returned back with an allow status:

User-agent: *
Disallow: /*/2007/$

7:10 pm on Apr 17, 2007 (gmt 0)

WebmasterWorld Senior Member ogletree is a WebmasterWorld Top Contributor of All Time 10+ Year Member



You can make a change in your header file or make a plugin.

To do this in your header insert this somewhere in the head tags. Before or after the Title tag is fine.

<?php
if (is_archive()){
echo "<meta name=\"robots\" content=\"noindex,follow\">\n";
}
?>

This will put noindex on Category, Author and Date based pages. That may be too much. Here [codex.wordpress.org] is a page that shows the different is_ codes you can try.

5:46 pm on May 3, 2007 (gmt 0)

5+ Year Member



Ogletree,

Question...This accomplished the same thing as the [seologs.com...] plug-in, correct?

I used the following code to archive only the index, page, posts:

<?php if(is_home() ¦¦ is_single() ¦¦ is_page()){
echo "<meta name=\"robots\" content=\"index,follow\">";
} else {
echo "<meta name=\"robots\" content=\"noindex,follow\">";
}?>

which seemed to work fine, then I found the plug-in and that seems to address the issue as well. Your thought on one vs. the other?

5:26 pm on May 5, 2007 (gmt 0)

WebmasterWorld Senior Member ogletree is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Yes it is very close to that plugin. I made comments on that site about my fix. I also blogged about it. I would have linked to it but thought we could not do that here.
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month