Forum Moderators: Robert Charlton & goodroi
/forum/viewtopic.php?f=1&t=4&view=unread
/forum/viewtopic.php?f=1&t=7&view=unread
/forum/viewtopic.php?f=1&t=5&view=unread
The content of these pages is always the same: the post does not exist (it is not a 404, but a message of the forum). They are my old tests.
So I have a big problem of duplicate content.
I thought to rename the forum directory into forum123 so all those pages will not exist anymore and inform google to remove it from its index.
Is this the correct way?
Please consider in my robots.txt I already have the row /forum/viewtopic.php.
Or do I have to do nothing and wait until google understand it has not to crawl those pages?
I think the google toolbar is the guilty
[edited by: tedster at 12:23 pm (utc) on Oct. 5, 2009]
[edit reason] switch to example.com - it cannot be owned [/edit]
BTW: GoogleBot and all other compliant Bots will access your robots.txt every visit, so you don't need to worry about submitting it to them or waiting for it to be found. As long as it's properly formatted any changes you make should be noticed and take effect the next time they spider any pages from your site.