Google 403 Error

[ControlZ]

I am experiencing an issue with a listing within Google's directory. The site in question has been live for over 10 years but recently we are experiencing a problem.

Google has the listing for our domain listed as thisdomain.com - without a "www." Not a huge issue or so I thought because I have an htaccess in place to force the user to http://www.thisdomain.com, however when a user clicks on the link within Google SOME users experience a 403/Forbidden error which I cannot figure out. I have checked Google Webmaster Tools, and server logs, all with no errors.

I have run this issue by numerous persons and no one seems to have a clue.

I beginning to think there is an issue with the htaccess file.

What is really strange is if a user enters thisdomain.com in their browser they can see the site, however these same users when clicking on the listing within Google get the 403 error. I myself am not able to duplicate this issue on any of my PCs.

[edited by: tedster at 3:29 pm (utc) on Nov 6, 2011]
[edit reason] stop the auto-linking for an example domain [/edit]

[aakk9999]

I have checked Google Webmaster Tools, and server logs, all with no errors

Unless I am missing something, if you don't see 403 in your logs then I am pretty certain that 403 is not returned by your server, hence it would not be .htaccess issue.

How do you know that some visitors gets 403 response if you do not see them in your logs? Were you told so by someone?

[lucy24]

What is really strange is if a user enters thisdomain.com in their browser they can see the site, however these same users when clicking on the listing within Google get the 403 error.

Obvious first question: have you blocked any aspect of google, whether by IP, UA or referer?

Does it happen to anyone who lives close enough that you can personally watch what happens when they click? Or anyone that can use a Live Headers-type extension to get more information to you?

When you say "no error in the logs", do you mean that there's no 403 getting logged at the exact time when your users report get locked out? A 403 would normally show up in both the regular logs and the error logs, with the same information in both. (Unless you are in Debug mode, error logs give the amazingly unhelpful "client refused by server configuration" or words to that effect.) But 500-class errors show up only in the Error Log.

[ControlZ]

aakk9999, yes I have been notified by several people that they are receiving 403 errors, otherwise I would have no clue.

lucy24, no I am not blocking Google, matter of fact there are several number one rankings for various keyword phrases.

I have screenshots of the error, but no headers.

There are absolutely no errors when a user tries but is refused access to the site.

[aakk9999]

There are absolutely no errors when a user tries but is refused access to the site.

It could be something their end then. Perhaps some proxy settings or similar?

[lucy24]

Oops, another obvious question to get out of the way. What form does the refusal take? That is, do they see your custom 403 page or something else?

[ControlZ]

aakk9999, I've been told by someone that works for a large hosting provider that there have been issues with a popular brand of routers and outdated firmware which have been none to cause 403 errors. I am not sure how much I believe this is the actual cause.

lucy24, no custom 403 page, just the default page generated by the browser which I have screenshot of.

[ControlZ]

aakk9999, I thought the same thing are you until I was told this is happening to people outside the main corporate office. I wish I knew more info on the complaints from users of the site (outside the corp office), but as is par for the course, I get emails saying the site is not working and then need to practically beat the sender over the head in order to get relevant info about the complaint.

[lucy24]

If it's a browser default error message then they're not even getting to your site. Is it the DNS error ("Check your typing" "Can you browser other sites?" etc.) or the time-out error ("I waited and waited and I'm giving up" typically after 1-2 minutes) or something else? This really doesn't sound like it could be a problem with your htaccess. At worst, they would then get a server-generated 500 error message.

Can you put up the exact text of the error message? A screenshot probably isn't necessary, unless it's got really distinctive visuals.

[ControlZ]

I just sent you a screenshot as well as the htaccess.

It's not a typing error because the link is coming from Google's search results page. They are not having issues with any other sites except their own, how ironic! : )

[ControlZ]

I've managed to figure out one piece of the puzzle. The screenshot shows a 403 error generated by a Windows server. Since the site is on a Linux box the error cannot be coming from my server. My guess is the error is being generated from their in-house server, but this does not explain why customers outside the main office are getting the same error, unless they are exaggerating for the purpose of getting me to react quicker. I have not seen a screenshot from outside the corporate office so I am in the dark as far as that issue is concerned.

[indyank]

check out for any DNS hacks as a server mismatch is serious...

[lucy24]

The error message says in full:

403 - Forbidden: Access is denied.

You do not have permission to view this directory or page using the credentials that you supplied.

I think this is seriously weird. It sounds more like a 401 than a 403. It will be very annoying if it turns out to be due to someone in an office going bonkers with firewalls.

[ControlZ]

Turns out the error was because the customer was on the same server as the domain and when they entered they clicked on a link (in Google) that did not contain the "www" the local server assumed they were trying to connect to their in-house server on the same domain, thus they were receiving prompts to log-in.

As far as their (the client) description that users outside the company were receiving errors when clicking on a link from within Google's search results, I found this to be an exaggerated claim. Since I got Google to re-index the site and modified the htaccess to force users to "http://www" the complaints have ceased.