Google not crawling website because of age verification script

[vatel]

Hello,
I recently built a website for a liqueur. Industry guidelines, and federal regulations require the website to have some sort of verification script that asks for age before allowing a visitor to view the pages. A simple click on a link stating the visitor is allowed to drink alcoholic beverages in his/her country is enough. I put such link on the homepage, and a php script allows the visitor to enter the site. No page can be reached before the visitor clicks on the verification link. If they get to an internal page before clicking on the link on the homepage, for instance directly typing the internal page url, they get redirected to the home page where they can "verify" their age by clicking on the link.
I submitted a sitemap to Google, but no internal page has been accessed, giving me an error in the Analytics site.
Is there a way to have those internal pages indexed by Goggle, and other search engines.
Sorry for the long post and thanks in advance.

[TheMadScientist]

If they get to an internal page before clicking on the link on the homepage, for instance directly typing the internal page url, they get redirected to the home page where they can "verify" their age by clicking on the link.

One of the first things I think when I read this is: If you can redirect to the home page, you can show the verification on the page and when the link is clicked show the information rather than redirecting them anywhere. If you can do one you should be able to do the other, so IMO it's in the interest of your site and visitors to rework the redirect to show the verification link on the page they were visiting rather than taking them somewhere else, and it may help with number 2...

I submitted a sitemap to Google, but no internal page has been accessed, giving me an error in the Analytics site.

My guess is Google hasn't clicked the link and is getting repeatedly redirected to the home page, so GBot cannot spider the pages.

Is there a way to have those internal pages indexed by Goggle, and other search engines.

IMO this is a case where you would use acceptable cloaking and a noarchive tag on the pages, so the bots are let through without the verification, they will not cache or archive the page so people have to visit the site and verify their age prior to viewing, but bots will still be able to access the content. I would not recommend a 'simple' user-agent cloak if you are supposed to have age verification in place, but would rather recommend a full-blown IP look-up cloak to make sure you are only letting SE bots through without the verification system kicking in.

[tedster]

There is a Google Webmaster Group thread on this topic where Google's Susan Moskwa replied:

What we recommend in this case is to serve it via JavaScript. That way users can see the age verification any time they try to access your content, but search engines that don't run JavaScript won't see the warning and will instead be able to see your content.

[groups.google.com...]

In essence, I read this as saying you may use scripting to achieve a "kind-of-cloaked" result in situations like this.

[TheMadScientist]

Regardless of the Google recommendation IMO it is much better if there is a legality involved to just plain cloak it, because Google is looking out for Google not you, and we know not everyone runs JS, which AFAIK is roughly 10%-12% of people on the Internet. If they offer to pick up your legal bills should they arise then by all means, do as they say, but personally, I wouldn't take the chance of getting in trouble to kiss Google's A**. But this is of course IMO only and others may see the situation differently.

In a 'non-critical' verification system, I would be more inclined to do what they are suggesting, but when it's something like alcohol and minors, which involves the ATF and possibly other such acronyms, I think safe is better than Google's self serving advice.

[tedster]

Another approach might be using a CSS div that overlays the web page on a high z-index -- until the user is verified, and then the visibility of the div gets changed. That way all the code can be served to googlebot exactly as with any user.

[TheMadScientist]

Good thinking tedster...

That would actually reverse the necessity of the JS, making it required to be turned on to view the rest of the content rather than the other way around and should ensure the verification is shown to all visitors not just those using JS.

[vatel]

As usual, an array of great suggestions! Thank you everybody for the quick response.
Tedster, I love the idea of the CSS div, do you have any idea where I could find some examples of it, either on this site or elsewhere? I am starting a Google search in the meanwhile.
Thanks

[TheMadScientist]

Just thinking about this one and I'm back to cloaking it, because the content is still accessible in the source code even with the div you have to close to see it in the browser window... Anyway, just my .05 (inflation) LOL.

[tedster]

I'd just check out the major providers of age restricted products and services. This is a challenge that is bound to be pre-solved in several different ways.

[vatel]

Thank you again. You are right Tedster anybody does it its own way. I'll go with Javascript and CSS and see what happens. I can still go back to my old php approach.
Thanks