Forum Moderators: phranque

Message Too Old, No Replies

Robots.txt Disallow by Accident, SERP's Gone - HELP!

Robots.txt Error

         

tommyjon

1:51 pm on Apr 30, 2012 (gmt 0)

10+ Year Member



Hi All -

I made a crucial mistake in my Robots.txt file which told Google not to crawl site and take it out of the SERP's. Took me about a week or so to find the issue but at that time my site was taken out of the SERP's. It has been about 5 days since correcting the Robots.txt file and the Google bot is back crawling the site and the Cache has updated. In Webmaster Tools the blocked pages are going down.

I still have not seen any of my rankings come back and all traffic is gone, my site has about 2000 pages, PR3 and 1.5 year old. Can anyone tell me if my site will ever appear back in the SERP's? What is a good time frame to let Google sort this out? Could this be permanent?

g1smd

7:48 pm on Apr 30, 2012 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



It can take a few days or so. Be patient.

tommyjon

11:33 pm on Apr 30, 2012 (gmt 0)

10+ Year Member



Thanks for the reply, been almost a week and still my home page isn't even ranking? I will try not to panic... I just hope it comes back and this is not permanent :(

g1smd

11:56 pm on Apr 30, 2012 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Run Xenu Linksleuth over your site and check what it finds.

sunnyujjawal

4:34 am on May 1, 2012 (gmt 0)

10+ Year Member



You can also try Google webmasters fetching feature .
Click on Diagnostics >> Fetch as Googlebot. Enter the URL's which you want to fetch . Click Fetch and the page will auto-refresh after 5 seconds and click "Submit to index".
I try this so many times and within 24 hours pages will be indexed.

Rest..Google never ignore Robots.txt...

lucy24

7:07 am on May 1, 2012 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Google never ignore Robots.txt

Well, googlebot by that name never ignores...

tommyjon

1:36 pm on May 1, 2012 (gmt 0)

10+ Year Member



I used Google Fetch, still didn't work. My pages are indexed but not showing in the serps. What kind of scares me is that I got un-natural links warning but seemed to recover fine. Then once I changed the Robots.txt my site was out of the serps. Fixed the Robots.txt and here we are almost a week later and my site still isn't in the serps?

Did I get penalty? Google is crawling.... How long would you wait before I should start to worry?