Welcome to WebmasterWorld Guest from

Message Too Old, No Replies

Panda and WordPress tags



10:56 am on Jul 16, 2012 (gmt 0)

10+ Year Member

Ive read a lot about Panda and WordPress sites... and I still have some doubt about WordPress tags....

Should we allow Google to index WordPress tags?

My site has over 4.000 post and over 11.000 tags.
If I search Google "site:example.com" mostly results in the first pages (90%) are WP tags.

If I search Google.com the exact title of one post, the first result is a tag -no the post- and also I get this:

In order to show you the most relevant results, we have omitted some entries very similar to the 3 already displayed.
If you like, you can repeat the search with the omitted results included.

I guess this mean the tags are been cosidered as duplicated content?

Actually my robot txt for the WordPress site is this:

User-agent: Mediapartners-Google

Sitemap: http://www.example.com/sitemap.xml.gz

User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /*/*/*/feed
Disallow: /*/*/*/trackback
Disallow: /*/*/*/attachment
Disallow: /author/
Disallow: /category/*/page
Disallow: /category/*/feed
Disallow: /category/*/*/page
Disallow: /category/*/*/feed
Disallow: /tag/*/page
Disallow: /tag/*/feed
Disallow: /page/
Disallow: /xmlrpc.php
Disallow: /*?s=

Where as you can see I only disallow for tags the duplicated ones:

Disallow: /tag/*/page
Disallow: /tag/*/feed

Should I just disallow all? like:
Disallow: /tag/

Also, and really dont know if it has relation with this problem, if I search example.com i get over 24,000 results indexed by Google.

If I see my WordPress Desktop:

All this is over 15,000 urls ... and indexed I see 24,000... yes I know there is Author pages, Date pages etc but not more than 1,000 I guess and also those should be restricted by above robot.txt... does it have any sense for you?


[edited by: tedster at 4:41 pm (utc) on Jul 16, 2012]
[edit reason] switch to example.com [/edit]


5:17 pm on Jul 16, 2012 (gmt 0)

10+ Year Member

Same deal when a spider cannot tell the difference between:

- cat.asp?id=124
- cat.asp?id=124&offset=30
- cat.asp?id=124&offset=60
- cat.asp?id=124&offset=90
- cat.asp?id=124&lang=english
- cat.asp?id=124&lang=french
- cat.asp?id=124&lang=german
- cat.asp?id=124&lang=spanish

Then you get an error report about duplicate pages.


5:18 pm on Jul 16, 2012 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Here's a video from Google's Matt Cutts that may help:

Do tag clouds help or hinder SEO? [youtube.com]


5:30 pm on Jul 16, 2012 (gmt 0)

IME, Google loves those tag pages for some reason. I had a problem with Panda on a site that had too many tags so I consolidated them and that has seemed to help.

(Edit: forgot to specify that I'm talking about WP)

[edited by: Panthro at 6:25 pm (utc) on Jul 16, 2012]


5:45 pm on Jul 16, 2012 (gmt 0)

WebmasterWorld Senior Member netmeg is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

I block all tags. Always. Just a personal preference. That's for WordPress and Magento.


6:29 pm on Jul 16, 2012 (gmt 0)

5+ Year Member

I block all tags.

Same here. We go a step farther and also create landing pages for each category as well, then noindex the Wordpress generated category pages also. We view auto-generated pages like WP categories and tags as pages for users who are already on the site, not to be indexed in search engines.

For tags, if the tag seems to warrant a page of its own that needs to be indexed, I will create a landing page, then wp_query all posts with that tag.

As far as the actual landing page goes, I just use a WordPress "Page", then use a "custom field" and some hand-rolled code in the functions.php file to drop in the catid or tagid.

My feeling is that bit of text needs to resolve to 1, and only 1, indexed url. I have no data saying that its better one way or another, but its my own pref and works for me.



8:54 pm on Jul 16, 2012 (gmt 0)

Instead of blocking them. I would advise you to do a "noindex,follow" meta tag. I wouldn't block feeds either.

Also ..why are you disallowing your themes directory? I think you'd want google to be able to access your js and css.


10:17 pm on Jul 16, 2012 (gmt 0)

5+ Year Member

Patterns and excessiveness of any thing is bad, tags are probably better off being Noindex, but keep them do follow!


10:34 pm on Jul 16, 2012 (gmt 0)

It really depends on how you use them. I use my tag pages as navigation elements, when there are important similarities within posts of different categories. I let Google index those, because they're valuable pages (with the addition of some into paragraphs).

On the other hand, I have seen blogs where people tag posts with a dozen meaningless (to a search engine) things. In those cases, I'd no index the lot.

Ultimately, the answer will vary depending on how tags are incorporated into the overall structure of your site.


7:55 pm on Jul 17, 2012 (gmt 0)

Some good information here thanks, ive always wondered about tags. Ive noindexed them but set them to follow. Ive allowed categoreis to eb index as I use them for nav.

If you do a site:example.com after 20+pages you can see the tag pages however, its just a title with no meta info. Ive always wondered if this had a negative effect.



8:16 pm on Jul 17, 2012 (gmt 0)

@chalky ..that happens with me when I have the noindex,follow tag + disallow in robots. If I remove the disallow, they noindex,follow tags don't get indexed. I don't know ...maybe a weird issue with Google ?


8:40 pm on Jul 17, 2012 (gmt 0)

@klark0 I think I may try the disallow, was your syntax

Disallow: /tag/*/




8:45 pm on Jul 17, 2012 (gmt 0)

10+ Year Member

I've no experience with WP, but I would noindex tag pages with one (or 3-5?) entries.

Featured Threads

Hot Threads This Week

Hot Threads This Month