Welcome to WebmasterWorld Guest from 54.196.49.210

Forum Moderators: Robert Charlton & aakk9999 & andy langton & goodroi

Message Too Old, No Replies

Panda and WordPress tags

     
10:56 am on Jul 16, 2012 (gmt 0)

Junior Member

10+ Year Member

joined:Apr 29, 2003
posts: 44
votes: 0


Ive read a lot about Panda and WordPress sites... and I still have some doubt about WordPress tags....

Should we allow Google to index WordPress tags?

My site has over 4.000 post and over 11.000 tags.
If I search Google "site:example.com" mostly results in the first pages (90%) are WP tags.

If I search Google.com the exact title of one post, the first result is a tag -no the post- and also I get this:

In order to show you the most relevant results, we have omitted some entries very similar to the 3 already displayed.
If you like, you can repeat the search with the omitted results included.


I guess this mean the tags are been cosidered as duplicated content?

Actually my robot txt for the WordPress site is this:

User-agent: Mediapartners-Google
Disallow:

Sitemap: http://www.example.com/sitemap.xml.gz

User-agent: *
Disallow: /cgi-bin/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /wp-content/themes/
Disallow: /wp-login.php
Disallow: /wp-register.php
Disallow: /*/*/*/feed
Disallow: /*/*/*/trackback
Disallow: /*/*/*/attachment
Disallow: /author/
Disallow: /category/*/page
Disallow: /category/*/feed
Disallow: /category/*/*/page
Disallow: /category/*/*/feed
Disallow: /tag/*/page
Disallow: /tag/*/feed
Disallow: /page/
Disallow: /xmlrpc.php
Disallow: /*?s=

Where as you can see I only disallow for tags the duplicated ones:

Disallow: /tag/*/page
Disallow: /tag/*/feed


Should I just disallow all? like:
Disallow: /tag/

Also, and really dont know if it has relation with this problem, if I search example.com i get over 24,000 results indexed by Google.

If I see my WordPress Desktop:
3.106Entries
1Pages
20Categories
11.642Tags


All this is over 15,000 urls ... and indexed I see 24,000... yes I know there is Author pages, Date pages etc but not more than 1,000 I guess and also those should be restricted by above robot.txt... does it have any sense for you?

Regards,

[edited by: tedster at 4:41 pm (utc) on Jul 16, 2012]
[edit reason] switch to example.com [/edit]

5:17 pm on July 16, 2012 (gmt 0)

Preferred Member

10+ Year Member Top Contributors Of The Month

joined:May 27, 2005
posts:428
votes: 3


Same deal when a spider cannot tell the difference between:

- cat.asp?id=124
- cat.asp?id=124&offset=30
- cat.asp?id=124&offset=60
- cat.asp?id=124&offset=90
- cat.asp?id=124&lang=english
- cat.asp?id=124&lang=french
- cat.asp?id=124&lang=german
- cat.asp?id=124&lang=spanish

Then you get an error report about duplicate pages.
5:18 pm on July 16, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


Here's a video from Google's Matt Cutts that may help:

Do tag clouds help or hinder SEO? [youtube.com]
5:30 pm on July 16, 2012 (gmt 0)

Preferred Member

5+ Year Member

joined:Jan 6, 2011
posts:472
votes: 1


IME, Google loves those tag pages for some reason. I had a problem with Panda on a site that had too many tags so I consolidated them and that has seemed to help.

(Edit: forgot to specify that I'm talking about WP)

[edited by: Panthro at 6:25 pm (utc) on Jul 16, 2012]

5:45 pm on July 16, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member netmeg is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Mar 30, 2005
posts:12906
votes: 194


I block all tags. Always. Just a personal preference. That's for WordPress and Magento.
6:29 pm on July 16, 2012 (gmt 0)

Preferred Member

5+ Year Member

joined:June 14, 2010
posts: 585
votes: 0


I block all tags.


Same here. We go a step farther and also create landing pages for each category as well, then noindex the Wordpress generated category pages also. We view auto-generated pages like WP categories and tags as pages for users who are already on the site, not to be indexed in search engines.

For tags, if the tag seems to warrant a page of its own that needs to be indexed, I will create a landing page, then wp_query all posts with that tag.

As far as the actual landing page goes, I just use a WordPress "Page", then use a "custom field" and some hand-rolled code in the functions.php file to drop in the catid or tagid.

My feeling is that bit of text needs to resolve to 1, and only 1, indexed url. I have no data saying that its better one way or another, but its my own pref and works for me.

MH
8:54 pm on July 16, 2012 (gmt 0)

Junior Member

joined:Jan 9, 2012
posts: 192
votes: 0


Instead of blocking them. I would advise you to do a "noindex,follow" meta tag. I wouldn't block feeds either.

Also ..why are you disallowing your themes directory? I think you'd want google to be able to access your js and css.
10:17 pm on July 16, 2012 (gmt 0)

Full Member

5+ Year Member

joined:Jan 9, 2007
posts:255
votes: 1


Patterns and excessiveness of any thing is bad, tags are probably better off being Noindex, but keep them do follow!
10:34 pm on July 16, 2012 (gmt 0)

Preferred Member

joined:Jan 12, 2012
posts:397
votes: 0


It really depends on how you use them. I use my tag pages as navigation elements, when there are important similarities within posts of different categories. I let Google index those, because they're valuable pages (with the addition of some into paragraphs).

On the other hand, I have seen blogs where people tag posts with a dozen meaningless (to a search engine) things. In those cases, I'd no index the lot.

Ultimately, the answer will vary depending on how tags are incorporated into the overall structure of your site.
7:55 pm on July 17, 2012 (gmt 0)

Full Member

5+ Year Member

joined:Feb 25, 2011
posts: 257
votes: 0


Some good information here thanks, ive always wondered about tags. Ive noindexed them but set them to follow. Ive allowed categoreis to eb index as I use them for nav.

If you do a site:example.com after 20+pages you can see the tag pages however, its just a title with no meta info. Ive always wondered if this had a negative effect.

Al
8:16 pm on July 17, 2012 (gmt 0)

Junior Member

joined:Jan 9, 2012
posts: 192
votes: 0


@chalky ..that happens with me when I have the noindex,follow tag + disallow in robots. If I remove the disallow, they noindex,follow tags don't get indexed. I don't know ...maybe a weird issue with Google ?
8:40 pm on July 17, 2012 (gmt 0)

Full Member

5+ Year Member

joined:Feb 25, 2011
posts: 257
votes: 0


@klark0 I think I may try the disallow, was your syntax

Disallow: /tag/*/

Thanks

Al
8:45 pm on July 17, 2012 (gmt 0)

Preferred Member

10+ Year Member

joined:Dec 12, 2004
posts:609
votes: 1


I've no experience with WP, but I would noindex tag pages with one (or 3-5?) entries.