Welcome to WebmasterWorld Guest from 54.167.0.111

Forum Moderators: open

Message Too Old, No Replies

Getting subscription pages indexed

     

musicales

8:26 am on Mar 15, 2003 (gmt 0)

10+ Year Member



I know this is a bit of a contradiction but has anyone found a way round this: getting google to index pages that are for members only; or, as I have on one site, pages that are cookied to allow only one page view per day.

Anyone tried, for example, looking out for googlebot in page code and serving it the full page, instead of redirecting to the login or whatever?

Many thanks

ddent

4:26 am on Mar 16, 2003 (gmt 0)

10+ Year Member



Webmasterworld seems to, and the way you want to do it is probably very similar to how it does it.

Server the full page to the google bot, but tell it not to make the cached page available. Otherwise people can just look at the google cache.

johnraphone

4:43 am on Mar 16, 2003 (gmt 0)

10+ Year Member



You may want to have it look for "googlebot.com" instead of just the user agent string because anyone can spoof that. And be sure it block the snippets like ddent said.

jomaxx

4:58 am on Mar 16, 2003 (gmt 0)

WebmasterWorld Senior Member jomaxx is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I really have a problem with this. Google is a SEARCH ENGINE, not a mechanism for you to advertise your members-only content.

What part of WebmasterWorld is spidered but is unavailable to non-members? I can seem to access it pretty well without signing in.

ddent

5:03 am on Mar 16, 2003 (gmt 0)

10+ Year Member



The forums.

Maybe its just due to my ISP (though its a pretty large ISP, and they would have other means of tracking abusers i.e. static hostnames), but for months many of the searches I did would turn up hits on webmasterworld, but when I clicked on them I was told I had to log in to read the message. Eventually I decided I may as well create an account. :)

BigDave

5:19 am on Mar 16, 2003 (gmt 0)

WebmasterWorld Senior Member bigdave is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Never had a problem with getting to WW forums before I became a member.

I'm with jomaxx. I would find it incredibly annoying if I came across a site doing this. You are wasting my time and reducing the quality of the search results.

rogerd

5:26 am on Mar 16, 2003 (gmt 0)

WebmasterWorld Administrator rogerd is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Although I've never observed this, I think Brett has at times required users from certain domains to log in to prevent abuse issues.

ddent

6:10 am on Mar 16, 2003 (gmt 0)

10+ Year Member



That would coincide with the message I was getting.

FWIW, hostnames (though not the associated IP addresses, however the reverse of the IP is updated constantly) ending in .bc.hsia.telus.net or .ab.hsia.telus.net will be constant and pretty much assured to be one customer. Most customers will have two hostnames, though some will have five.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month