Welcome to WebmasterWorld Guest from

Forum Moderators: rogerd & travelin cat

Message Too Old, No Replies

Disable Indexing of Comment Pages - Wordpress

10:17 pm on Nov 13, 2010 (gmt 0)

Full Member

5+ Year Member

joined:July 13, 2009
votes: 0

Lately Google has been indexing my comment pages. For example the URL would be url.com/page/comment-page-1/ which is a duplicate page of url.com/page/.

Would this "Disallow: /comment-page-*" fix my problem through robots.txt?
2:06 am on Nov 14, 2010 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
votes: 593

Suggest that all comment pages include the meta "noindex" as a belt and suspenders method as well.
5:13 pm on Feb 1, 2011 (gmt 0)

New User

5+ Year Member

joined:Feb 1, 2011
votes: 0

Ya, the "noindex" meta tag would help, but how do you add that to comment pages only? (without having to manually add)
10:04 pm on Feb 3, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member bwnbwn is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Oct 25, 2005
votes: 28

I have taken on a client using wordpress it is a nightmare to figure out all the duplicate urls being generated. I remember a post not long ago.
4:57 pm on Feb 8, 2011 (gmt 0)


WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
votes: 254

So this should only be dupe content for the first page, right? In which case, why not send a 301 for page/page-1 and redirect to the canonical URL?

Then get rid of the link on your site. The Wordpress SEO Pager plugin should, I think, get rid of the dupe for page-1
[seoegghead.com...] (in any case, it does for their blog).

And then for robot meta tags...

- headspace2 - this is what I use generally and it allows pretty good control of the robots meta - [wordpress.org...]

- AllInOneSEOPack - [wordpress.org...] - venerable plugin which might offer this, but I haven't used it in years.

- Joost de Valk's plugins
-- robots meta - [yoast.com...] - this one can definitely "Prevent indexing of just your comment feeds" though I'm not sure how that pertains here.
-- Wordpress SEO - [yoast.com...] (I think this is superceding the above)

- wpseo (commercial plugin) - [wpseo.org...] - promises to do more than the others, but those others do a lot. Unless it offers something truly unique and your problem is intractable by other means, I probably wouldn't go there, but I'm cheap.

While on the subject of dupe content, the bigger problem with Wordpress is typically in the archives. If you're starting from scratch, you can write custom excerpts that do not show on the content page, but only on listing pages. Also, you can add headers/intro text to your archive pages on a per category basis using Category Templates - [codex.wordpress.org...] . Then that intro text makes your archive pages unique (and makes them look more like a mini guide than a plan listing). Personally, this is one of those "if I had time, I would do this" things that I don't actually do.

- finally, a new plugin for finding and identifying weak content on your site, not really related to the subject at hand except in the sense of creating strong main pages and finding weak ones, is the SEO Content Control plugin - [linkstrasse.de...] - I'm not sure how long this has been around, but I think it's new in the last month.