homepage Welcome to WebmasterWorld Guest from 54.227.25.58
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld
Visit PubCon.com
Home / Forums Index / Google / Google News Archive
Forum Library, Charter, Moderator: open

Google News Archive Forum

    
Getting subscription pages indexed
musicales




msg:209249
 8:26 am on Mar 15, 2003 (gmt 0)

I know this is a bit of a contradiction but has anyone found a way round this: getting google to index pages that are for members only; or, as I have on one site, pages that are cookied to allow only one page view per day.

Anyone tried, for example, looking out for googlebot in page code and serving it the full page, instead of redirecting to the login or whatever?

Many thanks

 

ddent




msg:209250
 4:26 am on Mar 16, 2003 (gmt 0)

Webmasterworld seems to, and the way you want to do it is probably very similar to how it does it.

Server the full page to the google bot, but tell it not to make the cached page available. Otherwise people can just look at the google cache.

johnraphone




msg:209251
 4:43 am on Mar 16, 2003 (gmt 0)

You may want to have it look for "googlebot.com" instead of just the user agent string because anyone can spoof that. And be sure it block the snippets like ddent said.

jomaxx




msg:209252
 4:58 am on Mar 16, 2003 (gmt 0)

I really have a problem with this. Google is a SEARCH ENGINE, not a mechanism for you to advertise your members-only content.

What part of WebmasterWorld is spidered but is unavailable to non-members? I can seem to access it pretty well without signing in.

ddent




msg:209253
 5:03 am on Mar 16, 2003 (gmt 0)

The forums.

Maybe its just due to my ISP (though its a pretty large ISP, and they would have other means of tracking abusers i.e. static hostnames), but for months many of the searches I did would turn up hits on webmasterworld, but when I clicked on them I was told I had to log in to read the message. Eventually I decided I may as well create an account. :)

BigDave




msg:209254
 5:19 am on Mar 16, 2003 (gmt 0)

Never had a problem with getting to WW forums before I became a member.

I'm with jomaxx. I would find it incredibly annoying if I came across a site doing this. You are wasting my time and reducing the quality of the search results.

rogerd




msg:209255
 5:26 am on Mar 16, 2003 (gmt 0)

Although I've never observed this, I think Brett has at times required users from certain domains to log in to prevent abuse issues.

ddent




msg:209256
 6:10 am on Mar 16, 2003 (gmt 0)

That would coincide with the message I was getting.

FWIW, hostnames (though not the associated IP addresses, however the reverse of the IP is updated constantly) ending in .bc.hsia.telus.net or .ab.hsia.telus.net will be constant and pretty much assured to be one customer. Most customers will have two hostnames, though some will have five.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google News Archive
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved