Welcome to WebmasterWorld Guest from 34.204.191.31

Forum Moderators: Robert Charlton & goodroi

google webmaster error Mobile Usability

     
1:14 pm on Jul 6, 2019 (gmt 0)

New User

joined:July 6, 2019
posts: 1
votes: 0


Hi, I encountered a big problem with webmaster tools
My problem is that Google has indexed and indexed my internal WordPress folders
For example, some of the indexes are indexed by Google under the following subcategory
/wp-content/uploads/cache/
/wp-content/uploads/2017/
It should be noted that the contents of my file robots.txt are as follows
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Sitemap: /sitemap.xml
and input the code Options All -Indexes in htaccess
this screan shot error by google
please help me
[uupload.ir...]
7:37 am on July 7, 2019 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Nov 13, 2016
posts:1194
votes: 285


"Disallow" these folders in your robots.txt . When done, you can use the GSC to remove these pages from Google's index.
3:29 pm on July 7, 2019 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:4508
votes: 347


This:
and input the code Options All -Indexes in htaccess
has nothing to do with Google, it disables the (usually unwanted) behavior of serving up a list of files in any directory visited. To prevent Google's indexing of files you do not want indexed the ideal solution is to noindex those directories or to disallow crawling in robots.txt. Just be aware that disallowing in robots.txt prevents access to resources that Google would need to evaluate your site. If the googlebot is not allowed to see your .css, .js and image files they can't really know how it will appear for visitors. The workaround is to disallow directories and then allow types of files.

 
Disallow: /wp-admin/
Disallow: /wp-content/
Disallow: /wp-includes/
Allow: /wp-content/*/*.jpg
Allow: /wp-includes/css/
Allow: /wp-includes/js/
Allow: /*.jpg
Allow: /*.png
Allow: /*.css
Allow: /*.js

PS: no reason to allow anything within wp-admin folder such as "Allow: /wp-admin/admin-ajax.php" does.