homepage Welcome to WebmasterWorld Guest from 54.198.94.76
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
WMT: Blocking directory index pages via htaccess
webstuck




msg:4503804
 2:50 am on Oct 4, 2012 (gmt 0)

I actually posted this here, but a replier suggested I post in a google forum for better help. [webmasterworld.com...]

I recently learned that Google was indexing folder URLs on my site (e.g. /shop/) because when you go to these URLs, the server returns a list of files within the folder. I didn't want these types of URLs indexed by Google. So, I added Options -Indexes to my .htaccess file. Since I also have ErrorDocument 403 /404.htm within my .htaccess, these URLs now redirect to my 404 error page. Unfortunately, that caused Access Denied errors to appear within Google webmaster tools. I have a couple of questions:

One, is adding Options -Indexes to .htaccess the best way to deal with these URLs or is there a better option such as somehow causing the URLs to return 404 instead of 403 status codes?

Two, if I leave these URLs as returning 403 status codes, will the Access Denied errors go away within Google webmaster tools?

 

lucy24




msg:4503833
 4:46 am on Oct 4, 2012 (gmt 0)

if I leave these URLs as returning 403 status codes, will the Access Denied errors go away within Google webmaster tools?

No, but that's OK. From GWT's point of view, absolutely anything other than a 200 counts as an error. It doesn't automatically mean you did anything wrong or need to change anything. They're just drawing your attention to the situation in case you didn't intend for it to happen.

In some cases, a redirect may be appropriate. But only do it if you think that humans trying for /directory/ are looking for content that lives on some specific other page-- /directory/starthere.html, let's say.

I answered the Apache aspect of the question in the other forum.

g1smd




msg:4503845
 5:56 am on Oct 4, 2012 (gmt 0)

Returning 403 is fine. You're returning the right response.

The WMT report lets you know that Google has seen that response. Since that response is what you wanted, there's nothing else to do.

redirect to my 404 error page.

There should be no redirect. If requesting a folder results in a 301 or 302 response followed by a 403 or 404 response you have a big problem. The 403 or 404 response should be returned at the originally requested URL.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved