Welcome to WebmasterWorld Guest from 54.160.177.33

Forum Moderators: goodroi

Message Too Old, No Replies

How do I disallow bots from my Wordpress year Archive Folders

I guess this is a common question but I can't find an answer

     
1:35 am on Apr 16, 2007 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 22, 2003
posts:75
votes: 0


How do I disallow bots from spidering my archive folders in Wordpress?

Here is how my URL's are structured:

http://www.example.com/blog/index.php/2007/04/

I tried testing this in Webmaster Central, but it returned back with an allow status:

User-agent: *
Disallow: /*/2007/$

7:10 pm on Apr 17, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member ogletree is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 14, 2003
posts:4249
votes: 16


You can make a change in your header file or make a plugin.

To do this in your header insert this somewhere in the head tags. Before or after the Title tag is fine.

<?php
if (is_archive()){
echo "<meta name=\"robots\" content=\"noindex,follow\">\n";
}
?>

This will put noindex on Category, Author and Date based pages. That may be too much. Here [codex.wordpress.org] is a page that shows the different is_ codes you can try.

5:46 pm on May 3, 2007 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 22, 2005
posts:63
votes: 0


Ogletree,

Question...This accomplished the same thing as the [seologs.com...] plug-in, correct?

I used the following code to archive only the index, page, posts:

<?php if(is_home() ¦¦ is_single() ¦¦ is_page()){
echo "<meta name=\"robots\" content=\"index,follow\">";
} else {
echo "<meta name=\"robots\" content=\"noindex,follow\">";
}?>

which seemed to work fine, then I found the plug-in and that seems to address the issue as well. Your thought on one vs. the other?

5:26 pm on May 5, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member ogletree is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 14, 2003
posts:4249
votes: 16


Yes it is very close to that plugin. I made comments on that site about my fix. I also blogged about it. I would have linked to it but thought we could not do that here.