Google deindexed my pages after Access Denied Errors - 403 Response code in GWT

[sourabhrana]

Hello Fellow members,

From 9th may I am getting this error messages & these crawl errors is increasing daily. Google is not able to crawl my URLS & getting 403 response code & saying ACCESS Denied Errors in GWT. My all Indexed pages are de-indexed.

Why I am receiving this errors ? My website is working fine but why Google is not able to crawl my pages. PLEASE TELL ME what is the ISSUE, I need to resolve ASAP

on 9th may I got a message in GWT as well for "http://www.mysitename.co.uk/ Increase in authorization permission errors "

Google detected a significant increase in the number of URLs we were blocked from crawling due to authorization permission errors.


After this all problem started. Kindly tell what is the issue & how can I solve this.

[phranque]

welcome to WebmasterWorld, sourabhrana!


have you tried "fetch as googlebot" in GWT?
have you verified in your server access log that googlebot is getting 403 responses?
is googlebot getting a 403 for all requests or just some urls?
are any other user agents getting 403 responses or just googlebot?
perhaps there are clues in your server error log corresponding to the 403 responses you find in the access log.

[alika]

I've posted a similar problem here, though my site has not been deindexed, but traffic is way down

[webmasterworld.com...]

I have submitted Fetch as Google, even the wp-login page where all the authorization errors emanate (401 errors) and fetch status shows as "Success"

Changing robots.txt file to Disallow wp only reduced authorization errors from 687 to 673

Kilobytes downloaded per day remains very high since these authorization errors started. But search queries result show reduction by almost half. Traffic is also down by 30%

[sourabhrana]

Very very thanks @admin for welcoming me on this forum & feeling happy to the part of webmasterworld :)

"have you tried "fetch as googlebot" in GWT? " YES but not working. google not able to even fetch it.


"have you verified in your server access log that googlebot is getting 403 responses? " ... Till not bcoz I am not able to recognize the error. YES, I will now.


"is googlebot getting a 403 for all requests or just some urls? " - It is increasing day by day.
8th may - 246 errors
15th may - 2508 erros


"are any other user agents getting 403 responses or just googlebot? " -- only googlebot

perhaps there are clues in your server error log corresponding to the 403 responses you find in the access log. ---- I am also feeling like this .. but Not able to understand that why google is not able to crawl my site & Deindexed my all pages.

Message from google in 403 error urls

"Googlebot couldn't crawl your URL because your server either requires login to access the page, or is blocking Googlebot from accessing your site. "

[phranque]

check your error log file for possible errors related to the 403.

try accessing on a browser with a user agent switcher.