Welcome to WebmasterWorld Guest from 188.8.131.52
I had a quick question and couldn’t find it anywhere on the forum and was wondering if anyone would know please?
I am just wondering how you get all your forum pages indexed (like in webmasterworld.com) in the search engines without having the cache pages viewable as well? Also how the googlebot gets into the forum without having a cookie set? I would be keen to do this to my small forum that I keep as a paid only forum and I want to respect the members by not making all the information within it public, but still being able to generate traffic from search engines. Any help would be greatly appreciated!
Thanks in advanced
I don't know of any other way of making such pages indexable.
Hope this answers your thread.
And welcome to WebmasterWorld!
Also would this be considered cloaking as your not really giving the end user the actual page that would be indexed (assuming they don't have a cookie stored)?
Any help and info would be great.
There is also some info on
As you see there is no cookie included, as the crawlers are not run through browsers.
So if you spot a crawler - by IP and / or agent's name you can override you security and grant him access to some pages you want to index.
Hope this helps you
It is more like IP cloacking (which Google say is legal) as we are allowing users that have a cookie (at certain) IPs to view the pages and others will have to join to view.
Very hard to get the answer out of Google. Hopefulyl GoogleGuy could lend his 2 cents?