homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

Indexing restricted pages
aff developer

5+ Year Member

Msg#: 3140910 posted 12:10 am on Oct 31, 2006 (gmt 0)

Hi guys

I had a quick question and couldn’t find it anywhere on the forum and was wondering if anyone would know please?

I am just wondering how you get all your forum pages indexed (like in webmasterworld.com) in the search engines without having the cache pages viewable as well? Also how the googlebot gets into the forum without having a cookie set? I would be keen to do this to my small forum that I keep as a paid only forum and I want to respect the members by not making all the information within it public, but still being able to generate traffic from search engines. Any help would be greatly appreciated!

Thanks in advanced



WebmasterWorld Senior Member tedster us a WebmasterWorld Top Contributor of All Time 10+ Year Member

Msg#: 3140910 posted 6:12 pm on Oct 31, 2006 (gmt 0)

To keep your content out of Google's cache, place this meta tag in the <head> of the page for googlebot:

<meta name="googlebot" content="noarchive">

I am not technically knowledgable enough to know how the cookies thing works here with Google. All I know is what's in our threads, from the time last year when Brett went to war against rogue bots. That's when all these changes took place. Our current situation is the result of MUCH work on his part -- I doubt that the precise details will be public, or else the rogue bots would find a way around the barriers that Brett built to protect our server.


Note that we do not use a database here -- these forums run on flat files. So I doubt that Brett's final solution, even if made fully public, would be applicable to your situation.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved