Msg#: 4455412 posted 9:27 pm on May 19, 2012 (gmt 0)
I have category pages in the sitemap(s) I submit to Google's Webmaster Tools. Some of these categories span multiple pages. I've only been including the first page of each category in my sitemap but I'm wondering if I should also be including the paginated pages?
I use Google's recommendation of the rel/prev tags to let Google know it's a series - so it's ok for Google to be spidering these pages. Some categories have quite a few pages (100) and while I'm sure Google can find the pages (the site architecture makes sure the pages are crawlable) I'm just wondering if I should be helping by pointing them out. Now, I don't see those paginated pages in the index - but if I'm using the rel/prev tags, would I necessarily find them or would Google only return the main page of the category?
This is a site that's been hit by Panda (1.0 and every iteration since) primarily on the category pages (I'm working on that whole thin content issue) so I'm also open to suggestions about using rel/prev for pagination vs noindexing those pages. I don't want to use the "view all" option that Google also suggests because some of the categories are just too big to make that feasible (and it's that much easier for scrapers to get at the content).
Msg#: 4455412 posted 2:04 am on May 20, 2012 (gmt 0)
The sitemap is always an optional extra. If everything on your site is linked from somewhere else on the site-- or somewhere on someone else's site-- google will find it anyway. Sitemaps are only necessary if you have a section of your site that isn't linked from anywhere, but you still want search engines to index it. (Query: Under what circumstances would this happen?)
Conversely: Once you've got something on a sitemap, google will remember it forever. Doesn't matter if the page disappeared in 2006 and you've created eighteen new sitemaps since then. Google will still crawl it periodically, and will list it as a Crawl Error and claim that it's On Sitemap.