The main problem is that your code does not match your stated requirements.
If you want to redirect the "folder-only" URL, then you must detect requests for "/folder/ followed by nothing." You may wish to detect requests for "/folder followed by nothing" as well (no trailing slash) in order to avoid mod_dir doing a redirect to add that slash.
The regex pattern for that would be ^/folder/?$
If you want to redirect if the requested image does not exist, then you must check for file-exists. Otherwise, the usual 404-Not Found error handling will be invoked for missing images.
In your top-level .htaccess, this would give:
# Redirect if subfolder index is requested
RewriteRule ^subfolder/?$ http://subdomain.example.com/ [R=301,L]
#
# Redirect if non-existent file in subfolder is requested
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^subfolder/(.+)$ http://subdomain.example.com/$1 [R=301,L]
Note that we usually do not recommend redirecting requests for non-existent resources unless you have absolutely no concern for search engine ranking of those resources or that pages that link to them. If SE ranking is a concern, then using the server's normal 404 response to provide a page with a link to the desired 'destination' for users (both people and SE robots) making bad requests is to be strongly preferred.
Further, the more restrictive you can make the second rule above, the better. File- and directory-exists checks invoke the operating system's file-handler and may result in a read of the physical disk if the cached filesystem state is 'dirty' (not up to date). This can be thousands of times slower than executing a 'normal' mod_rewrite directive, and of course, it can also beat your hard drive to death. Therefore, enhancing the rule by using a more-specific rule pattern or by adding an exclusionary RewriteCond at the top to only check the disk for certain filetypes --or equivalently, to NOT check the disk for certain other filetypes-- is a good idea...
Note that RewriteConds are not evaluated unless the RewriteRule pattern matches (see the mod_rewrite documentation).
This code can be 'ported' back into /subfolder/.htacess if you like. Having two alternatives, I simply picked the one that resulted in a more-specific example.
Jim
