Google indexes my site's forums immediately, but nothing else

Actual site content ignored in favour of message board postings

[gibbergibber]

My site runs on a CMS where I have various sets of articles forming the bulk of the site, and some message boards (also running under the same CMS) where people can discuss those articles and related topics. It's all hosted on one server under one directory.

Over the past month or so Google has indexed the contents of the boards within days of their appearance and the site gets most of its traffic that way.

At the same time, the articles (what I regard as the true meat of my site) have been left untouched by Google for many months. Google indexes the article index, but not the article pages themselves.

What's going on here? Why is Googlebot so much keener on the boards than on the articles? Surely it would make more sense the other way round?

[webdude]

New and fresh content may be the answwer. Are your "meat" pages being changed, added to, updated regularily? You might want to give that a try. Another question... When searching for your site, do the "meat" pages list higher then the forums? might be a clue there. If not, I don't see what the problem is. G is always looking for fresh stuff. My forums get heavily crawled almaost daily but my "meat" pages get crawled once a week BUT my "meat" pages always outrank my forums and pop up very high in the SERPs.

[edited by: webdude at 5:25 pm (utc) on Aug. 23, 2006]

[webdude]

After re-reading your post... make sure all your article pages have different titles and meta-descriptions. That might help.

[gibbergibber]

-New and fresh content may be the answwer. Are your "meat" pages being changed, added to, updated regularily?-

Well, the individual articles aren't meant to be updated, once they're done they're just done. But the same could be said of the posts on the message board.

And here's another weird thing: Google lists each and every member's profile page, even though there's nothing really on them except the members' names.

--Another question... When searching for your site, do the "meat" pages list higher then the forums?--

Most of the meat pages don't appear at all! That's the problem, I could cope with them appearing lower but they're just not even listed.

But most of the forum pages appear in results and ALL of the member profile pages too.

The articles are placed in sections for ease of reference and the index pages of each section shows up, but none of the articles themselves.

In short, it's like Google indexes EVERYTHING on my site except the pages I want it to! :-)

-- After re-reading your post... make sure all your article pages have different titles and meta-descriptions. That might help. --

Yeah. I can't alter the meta-description because of the way the CMS works, but I made sure that each page has its own title.

[webdude]

Well gibbergibber, I think you need to get a way to alter your descriptions. I have always tried to alter the title and description for every page. I had a site that was at least 80% supplemental until I made these changes. It seems altering just content may not be enough.

Good thread here...

[webmasterworld.com...]

[gibbergibber]

-- Well gibbergibber, I think you need to get a way to alter your descriptions. I have always tried to alter the title and description for every page. I had a site that was at least 80% supplemental until I made these changes. It seems altering just content may not be enough. --

Why would that affect just the articles and not the forums, profiles or index pages though? That's what's puzzling me, it's like it's filtering out anything with any actual created-for-the-site content on it but indexing everything else.

Is it possible there's a hierarchy thing here, that the articles are buried one level lower than everything else so Google ignores them?

[webdude]

Not sure. I would view the source of a sample of pages to see if you can see if there is any reason. Pay particular attention to what is duplicated from page to page and what is not. I have had a very similar problem, but just the opposite. I could never get my forum pages indexed. All the rest of my site would index just fine. All my forum pages would end up in the supplemental index. I rewrote the forum to include things like the topic in the title and description - Also added an H tag for the topic on each page. Took a while, but now I have thousands of pages showing up.

mmmmmm

Really not sure of the hiearchy thing. Maybe someone else coould help you there. I really don't use folder hiearchies on any of my sites.

[gibbergibber]

Okay, I think I've solved this. For some reason Google will instantly index whatever I link to on my front page directly.

That's why the forum posts appear in the index so quickly, because there's a "latest posts" list of links on the front page. When I added a "latest articles" list too, they got indexed by Google within a couple of days.

It seems front page links help an awful lot on my site for some reason, and anything that isn't linked to on the front page is ignored.

[webdude]

Well that makes sense. Glad you got it figured out!