Considering IP delivery for protected content

[Itagnc]

I have a client with 10,000 unique pages protected by user/pass login. This login is required for reporting and tracking, but the client would like to have these pages crawled and indexed. My first recommendation was to remove any registration barriers that may prevent bots from crawling, but this is not an option.

So that leaves us with IP delivery to get these pages crawled. My only reservation is that the user will be met with a registration form, or prompt for user/pass when clicking through from an SE. It's a frustrating user experience, but it is the only way I know to get the content crawled.

Once again, the bots will see the same page a user sees, only the page is blocked by login.

Other than providing an annoying user experience, can anyone highlight additional risks from using this method? Would a search engine ban based on this technique?

[bhartzer]

A lot of major news media outlets are doing it, so I don't see why you can't. It's obviously working for them.

[Itagnc]

It seems those outlets have a deal with the engines - DallasNews.com comes to mind. Wonder how Google feels about "non-partners" using the same technique. I'm also concerned about ticking off our client's visitors - there's no question that many people are put off by this.

[bhartzer]

So, the issue sounds as if it's not an IP delivery issue but a user experience issue and whether or not to make them register before they see content.

[Itagnc]

That would be accurate. So since most engines don't mind cloaked content if it is the same content as what a user sees, does that cloud the definition of 'approved" cloaking when the user/pass page is thrown into the mix?

[justguy]

We have done this on 4 vertical portals for 2 years. Even sent Google a note saying what we were doing and why. Never got a reply but did keep the email incase.

We auto-fill on a weekly basis our IP list with the tier 1 and tier 2 search engine crawler IPs.

No penalty and even got our news onto Googlenews despite it being often member-only material.