I want to stop Google indexing my "monthly" pages in my blog

...should i use robots.txt?

[sandpetra]

Can anyone help?

I want to stop Google indexing my "monthly" pages in my blog...should i use robots.txt?

Is this the best method?

I am going to remove these links from my blog but should I add a robors.txt first?

[tedster]

robots.txt is the way to go.

[sandpetra]

Thanks Tedster. I read the wordpress post in the sticky but still a bit bewildered with the myriad of options. Could you throw me a bone?

[tedster]

Excuse me for jumping to a conclusion - I'm not all that conversant with WordPress. From poking around, I see that many people are advising that you edit the header.php file on your the templates so that a robots meta tag of "noindex,follow" is included in each head section.

However, a robots.txt rule could also do the job you asked about -- but you'd need to remember to update it every year. Assuming your blog is at the root of your domain:

User-agent: *
Disallow: /2006/
Disallow: /2007/

[ogletree]

You can also use a plugin for that or just create your own it is quite easy. We talked about this a month ago.

[webmasterworld.com...]

[jk3210]

Let's say you have a blog that you HAVE done all the seo fixes to like disallowing "categories" etc, so you don't have a dup content problem --there's only one url per post.

You make a post [post123.php] and it gets spidered by Google. A week later [post123.php] has scrolled off of the index page.

Question: Where is that post now getting its link juice from? Doesn't it simply become an orphaned url?

[sandpetra]

I was going to keep categories.

[jk3210]

I was asking the question for my own knowledge, really. I've been wondering about it for a while now. I've seen numerous high-profile blogs that go gray-bar or PR0 once you get a few months deep into the site.

I'm just curious.

[jay5r]

I'd go with 'noindex' with a robots meta tag. You want your monthly pages to be crawlable so all your posts are discoverable... With robots.txt the monthly pages aren't crawlable.

[ogletree]

jk3210 the method that I talked about only disallows pages not links. Google still loads the page and gets the links off of it. When you do noindex, follow you get credit for all links on your site. I did this to my blog and I have very few pages in the sup index and my rankings have improved a lot. I highly recommend cleaning up wp dup pages.