Forum Moderators: open
I've found it increasingly fruitful to integrate message boards into my website, so instead of just publishing an article as a webpage I might post it to my site's message board system and link to this post from my site's main page.
My question is, how does Google decide whether or not to catalogue posts from message boards?
I can understand that the bots would want to avoid trawling through every script-based link ad infinitum. However, assuming there's a direct link to a message board post from an already catalogued HTML page, how likely is Google to index the post?
I've noticed that it's quite willing to index the indices of message boards, for example:
[mydomainname...]
but it hasn't so far (after several updates which managed the rest of my site) indexed any individual threads I've linked to.
I'm also trying out a PHP board system, and was wondering if this would be treated differently to a perl cgi one.
-gibbergibber
The vanilla version used session ID's, which google avoids. However, after adding in a mod which recognises the googlebot and serves its a SID free board, everything's gone swimmingly.
It seems quite happy to index every daft little page - I've seen googlebot simultaneously read 25 identical search pages having followed the links from the actual threads. About 390 pages went in the first time round, and I'm expecting a similar number to reach the index soon - there were about 30 Googlebot 'guests' online on my forum for most of Thursday.
Roddy
There's good content on message boards (okay, so it's in there with 99.9% rubbish sometimes!), it's a shame Google won't let you search it the way they let you search Usenet through Google Groups.
Google Guy, any chance of a Google tab devoted to message boards? (hopeful look)
-gibbergibber
You have to be considerate when "crafting urls". The consensus is that you should try to have no more than 2 Parameters in the url. With more parameters the links are likely to be ignored. You should also avoid having "id=" as a parametername (for gbots fear of session-ids). You should also be careful to not serve session-ids to all users. Remember that most dynamic applications use cookies by default, and fall back to session-ids only when cookies are disabled. Since Googlebot does not use cookies, it will get a session-id, even if you as a user might not be aware of it (if you have cookies enabled).
You can get every Forumsoftware to be spidered, it's just a matter knowing what and how to do it. You should also think carefully about which parts you do want and which you do not want to be indexed (use meta-tags). It's an important decision.
[edited by: rcjordan at 8:34 pm (utc) on July 31, 2003]
[edit reason] sorry, no references to your site, please. [/edit]
Please! Cgi posts aren't indexed! Do a search for wwwboard/messages
My hypothesis is that googlebot discriminates between those who have something to gain online (SEO) and joe the plumber's board where people dont use signatures for search engine benefits.
Nowadays I am using vBulletin. Google should theoretically be able to index the main page and the main category page and threads. But not the following page(s) since more than 2 parameters are used to define some settings.
Actually, Google has only indexed the main category pages at my forum and none of the threads. Any good hacks for vBulletin? The main page of my forum is PR5.
Does it have session ID in the URL?
Yes= You don't get listed.
No= You do get listed.
With VBulletin boards I went almsot two years with the Google Death Penelty then I took the session ID out of the URL in January and right now there are 15,600 URLs indexed.
:::i heard that Google will spider your dynamic pages, only if your PR is high enough. is it true?
Nope. I got a second vBulletin site, a new site, and before even registering a PR or back-links, it's indexing threads and boards.
I am currently using 2.2.9 and I know 2.3.0 is out for a while. Do you use the beta of 3.0?
