Googlebot is indexing "Post Reply" Link

phpBB2

[trillianjedi]

Got a site with phpBB2 running as the forum engine - mod-rewritten to allow the pages to be indexed.

Googlebot seems to be "clicking" the "Post Reply" button (well I guess it is a link) and indexing a reply page (where a user would type a reply or a post just like I'm doing now).

Is there an easy fix?

We could mod-rewrite to make the reply link "domain.com/forum/reply/blah.php" and then block the "/forum/reply/" directory in robots.txt, but it seems to me this must be a standard phpBB2 problem that everyone has and there's likely to be a simpler answer?

Thanks,

TJ

[sidyadav]

I've heard somewhere that you need to cloak Google's user-agent and disable session id's.

Alternatively, you can put the no-index meta tag in the header of the reply link.

Here's a thread about it, discussed a long time ago:
[webmasterworld.com...]

Sdi

[trillianjedi]

Thanks Sid - we actually don't use SID's.

I think the problem relates to Google improving it's ability to spider dynamic URL's with multiple variables, as I've never had this before with phpBB2 - it's a recent thing.

Example of what's being spidered:-

w*ww.domain.com/index.php?name=PNphpBB2&file=posting&mode=reply&t=4886

This is then being indexed in google.

TJ

[rogerd]

I've had this same problem with shopping cart links, i.e., Googlebot follows the "buy" button link. Even placing the cart page in the robots.txt file lets the link get picked up and listed in the page count; the more permanent solution is to let Googlebot spider the page and find the NOINDEX tag.

I don't mind Googlebot clicking all those "buy me now" buttons, but I wish they'd give him enough of a budget to actually buy something once in a while. Maybe after their IPO he'll get his own credit card. ;)

[trillianjedi]

the more permanent solution is to let Googlebot spider the page and find the NOINDEX tag.

Roger, that's so simple, but I didn't even think of that....!

Many thanks.

So I guess we need to find the section of the phpBB2 script that builds the "reply" page and insert the NOINDEX tag into the header.

Has anybody done this already - would sure save us an awful lot of time.

Thanks,

TJ

[sidyadav]

> Has anybody done this already - would sure save us an awful lot of time.

Yeah, I've done it with one of my sites running PostNuke - works great.

Sid

[trillianjedi]

Yeah, I've done it with one of my sites running PostNuke - works great.

Um, actually - I was fishing for the code... I know it's a bit cheeky, but this site that it's causing a problem on is hobby related, and doesn't actually generate any money.

Would you mind sharing - or maybe point me in the right direction?

Thanks,

TJ