Welcome to WebmasterWorld Guest from 54.205.0.26

Forum Moderators: rogerd & travelin cat

Message Too Old, No Replies

Search Engine access to WP-ADMIN

     
1:01 am on May 6, 2014 (gmt 0)

Preferred Member

5+ Year Member Top Contributors Of The Month

joined:May 29, 2007
posts: 627
votes: 45


Do search engines, Google in particular, need access to my wp-admin area? They demoted me from 5 stars for SEO to 2 stars because they cannot access that area. If they need access, how do I allow it? I use Yoast. Thanks for your help.
1:13 am on May 6, 2014 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator travelin_cat is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Feb 28, 2004
posts: 3186
votes: 7


The search engines do not need access to your wp-admin directory. In fact, it's good practice to password protect them.

Which stars are you referring to? And how do you correlate the two?
6:41 am on May 6, 2014 (gmt 0)

Preferred Member

5+ Year Member Top Contributors Of The Month

joined:May 29, 2007
posts: 627
votes: 45


The "stars" are the five blue dots in the Google AdSense dashboard. I have had five blue dots until I found the following message: Crawler errors - Robot denied for [..........] and [.......]
6:47 am on May 6, 2014 (gmt 0)

Preferred Member

5+ Year Member Top Contributors Of The Month

joined:May 29, 2007
posts: 627
votes: 45


Well, that didn't work out well, did it.

robot denied for:

wp-admin
wp-admin/customize.php
11:32 am on May 6, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lorax is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Mar 31, 2002
posts:7575
votes: 0


Is Google Bot denied via the htaccess?
5:56 pm on May 6, 2014 (gmt 0)

Preferred Member

5+ Year Member Top Contributors Of The Month

joined:May 29, 2007
posts: 627
votes: 45


No, Google Bot is indexing the site. It seems their problem is having no access to the admin area.
6:37 pm on May 6, 2014 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:2931
votes: 95


You have the /wp-admin/ folder Disallowed in robots.txt, right? That is all you really need to do - and ignore their pleas, there is nothing in there they need. They will get over it.
9:08 pm on May 6, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:13265
votes: 363


No, Google Bot is indexing the site.

I thought he meant, is the googlebot blocked from that specific directory. But it shouldn't make any difference, because if the robots.txt block is correctly worded-- which it obviously is-- then search engines will never ask for the page in the first place.

What worries me more is why they're asking for pages in this directory. Surely you don't have visible links to private content?
12:36 pm on May 7, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lorax is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Mar 31, 2002
posts:7575
votes: 0


@not2easy @lucy24 - that's what I'm wondering. Robots should get rid of the issue but I've never seen Googlebot even request it on my sites. Maybe the sitemap file includes it?
4:04 pm on May 7, 2014 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
posts:2931
votes: 95


Oh, I just remembered something - after the issue of the responsive version of WP, when auto-updates started, I saw a similar weird note in the AdSense UI - Seems they wanted to access wp-admin/css/colors/*.scss. Because this "error" is not due to Googlebot or any search engines, but an evaluation process within AdSense, it really isn't controlled in robots.txt or GWT.

I'm not saying that is the issue for the OP, but given that this is about Stars in the AdSense interface it could be. They seem to place a good deal of importance on responsive issues. To see what specific files they want to look at in wp-admin, all you need to do is click on the warning. The problem is not related to search engines or crawling, only to pleasing the responsiveness checking component of AdSense.