Blocking directory index pages via htaccess

[webstuck]

I recently learned that Google was indexing folder URLs on my site (e.g. /shop/) because when you go to these URLs, the server returns a list of files within the folder. I didn't want these types of URLs indexed by Google. So, I added Options -Indexes to my .htaccess file. Since I also have ErrorDocument 403 /404.htm within my .htaccess, these URLs now redirect to my 404 error page. Unfortunately, that caused Access Denied errors to appear within Google webmaster tools. I have a couple of questions:

One, is adding Options -Indexes to .htaccess the best way to deal with these URLs or is there a better option such as somehow causing the URLs to return 404 instead of 403 status codes?

Two, if I leave these URLs as returning 403 status codes, will the Access Denied errors go away within Google webmaster tools?

[wilderness]

You cannot have it both ways.
You either allow indexes or you don't.

There are multiple Google forums here at Webmaster World that could more readily address GWT than the Apache Forum.

[lucy24]

is adding Options -Indexes to .htaccess the best way to deal with these URLs

Yes. Assuming you don't want humans pawing through the files either. Most people in most circumstances turn off auto-indexing.

these URLs now redirect to my 404 error page

Fortunately you are wrong about this. They don't redirect-- which would be calamitous-- they simply get served the same page as a 404. Or, in the case of robots, they don't. As soon as they see the 403, they slink away unsatisfied.

The rest of the question really does belong in a g### forum.

[phranque]

One, is adding Options -Indexes to .htaccess the best way to deal with these URLs or is there a better option such as somehow causing the URLs to return 404 instead of 403 status codes?

Two, if I leave these URLs as returning 403 status codes, will the Access Denied errors go away within Google webmaster tools?

Options -Indexes is a good idea in any case.

since those directories were already indexed it would probably be better to send a 410 Gone response instead of a 403 Forbidden status code.
you can do that in .htaccess using mad-rewrite and the [G] flag

google often requests previously indexed urls for a long time, so as long as you are forbidding those requests GWT is likely to report them as access denied.
an "error" as reported in GWT is not necessarily a bad thing unless you didn't want that "error" to happen.