Welcome to WebmasterWorld Guest from 54.227.47.228

Forum Moderators: goodroi

Message Too Old, No Replies

How do I disallow bots from my Wordpress year Archive Folders

I guess this is a common question but I can't find an answer

     
1:35 am on Apr 16, 2007 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 22, 2003
posts:75
votes: 0


How do I disallow bots from spidering my archive folders in Wordpress?

Here is how my URL's are structured:

http://www.example.com/blog/index.php/2007/04/

I tried testing this in Webmaster Central, but it returned back with an allow status:

User-agent: *
Disallow: /*/2007/$

7:10 pm on Apr 17, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member ogletree is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 14, 2003
posts:4267
votes: 21


You can make a change in your header file or make a plugin.

To do this in your header insert this somewhere in the head tags. Before or after the Title tag is fine.

<?php
if (is_archive()){
echo "<meta name=\"robots\" content=\"noindex,follow\">\n";
}
?>

This will put noindex on Category, Author and Date based pages. That may be too much. Here [codex.wordpress.org] is a page that shows the different is_ codes you can try.

5:46 pm on May 3, 2007 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 22, 2005
posts:63
votes: 0


Ogletree,

Question...This accomplished the same thing as the [seologs.com...] plug-in, correct?

I used the following code to archive only the index, page, posts:

<?php if(is_home() ¦¦ is_single() ¦¦ is_page()){
echo "<meta name=\"robots\" content=\"index,follow\">";
} else {
echo "<meta name=\"robots\" content=\"noindex,follow\">";
}?>

which seemed to work fine, then I found the plug-in and that seems to address the issue as well. Your thought on one vs. the other?

5:26 pm on May 5, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member ogletree is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Apr 14, 2003
posts:4267
votes: 21


Yes it is very close to that plugin. I made comments on that site about my fix. I also blogged about it. I would have linked to it but thought we could not do that here.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members