homepage Welcome to WebmasterWorld Guest from 54.205.241.107
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

    
Blocked pages listed when I run site: command
member22

5+ Year Member



 
Msg#: 4637300 posted 10:02 am on Jan 14, 2014 (gmt 0)

In the omitted results of google when I type the site command I see listed pages whose folder is robot in my robots.txt such as /module / images/ components / administrator

What can I do about that ? remove those one by one as they appear with the url removal too, remove the robots.txt, 410 those ?

Can pages like that hurt my SEO

Thank you,

 

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4637300 posted 1:00 pm on Jan 16, 2014 (gmt 0)

How many pages are you talking about and how is it impacting your business?

member22

5+ Year Member



 
Msg#: 4637300 posted 1:15 pm on Jan 16, 2014 (gmt 0)

Impact : my rankings dropped considerably.

I am seeing 85 pages when I type the site command and the webmaster tools shows 85 too when my website is only 39 pages.

By the way the number of pages in the webmaster tools keeps increasing lately for 67 to 85 in the last one month and half and in the last year I haven't added any new pages.

Why is that ? any to figure which pages google has in its index that could be hurting me because I have the feeling there are some.

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4637300 posted 1:36 pm on Jan 16, 2014 (gmt 0)

Google will display the url of pages blocked by robots.txt. They may even show urls that do not exist since they can not crawl the page if it is blocked by robots.txt to discover that it does not exist. This will not impact rankings.

Are these pages blocked by robots.txt or has Google crawled them?

member22

5+ Year Member



 
Msg#: 4637300 posted 2:01 pm on Jan 16, 2014 (gmt 0)

That is true the 2 urls it is currently showing in omitted results are url that I blocked in the robots.txt.

The reason I blocked those is because google crawled those url during the bug I had a year ago.

Should i remove the robots.txt let google crawl then, them remove those once they appear and finally re-block those ?

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4637300 posted 4:39 pm on Jan 16, 2014 (gmt 0)

Blocking Google from crawling one page will not lower the rankings of your other pages with a few rare exceptions. I would look elsewhere to explain the rankings drop instead of spending time on non-essential pages blocked with robots.txt. That is more of a non-vital cosmetic issue.

member22

5+ Year Member



 
Msg#: 4637300 posted 5:25 pm on Jan 16, 2014 (gmt 0)

I see but where would you look ?

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4637300 posted 7:18 pm on Jan 16, 2014 (gmt 0)

This will answer your question [webmasterworld.com...]

Another option is to go here [webmasterworld.com...]

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved