My website ranking dropped!

after i create a wrong robotx.txt...

[virtualreality]

My website is a content website. It was doing great up to Thursday when by mistake i I did disallow / instead of disallow. So the next few days i noticed even bigger drop in SE.

I fixed this issue in Sunday but since then my organic ranking has not improved at all. On the contrary it is very bad. I have RSS, XML sitemap, txt sitemap and html sitemap and i add content all the time. So i wonder what must be wrong? Any ideas would be greatly appreciates!

Also if in my meta I use <meta name="robots" content="index,follow"> do I still need to use a robots.txt file:

User-agent: *
Disallow:

Thanks
Sisie

[jeffposaka]

There are no files or folders that you need to block? It is hard to believe.

I think meta tag is not needed across the site if you are on top of the robots.txt file.

[jdMorgan]

If you wish to allow robots to access everything, then you don't need the <meta name="robots" tag at all, and your robots.txt file can be blank. I recommend that even if it is blank, you should still have the file there, since this will prevent many 404 errors in your logs from robots trying to fetch it if it's missing.

It may take several more days or even a couple of weeks for your site to recover mostly, and longer to recover fully. Always review your work carefully and test it. There are several files in which you cannot afford to make mistakes, and robots.txt is one of them...

Jim

[Senthil]

I had the same experience but I got reindexed within few days...

Better update/create your Google Sitemap/XML file and resubmit it to Google Webmasters Tools...

[catman]

Google stopped indexing my site on July 11, and I found a robots.txt file that I did not place on the site that appeared to be blocking all search engines. We removed the file altogether. Is it possible that my host added this file, and will Google now begin indexing the site with no robots.txt in place? The site is: <snip>

Thanks for your help.

[edited by: goodroi at 3:32 am (utc) on July 29, 2008]
[edit reason] Please no specific URLs [/edit]

[JamieBrown]

I had the same experience but I got reindexed within few days...

I've seen it take a few weeks. I think it depends on how popular your site was to start with.

I would recommend:
1) Resubmit your sitemap
2) Do a bit of good old fashioned link building

What is your webmaster tools analysis looking like right now? Does it have the latest robots.txt or is it showing the old version?

[catman]

Thanks for your help. The analysis reads 404 not found. Does this mean I need to create a robots.txt page and leave it blank, or will the search engines index the page anyway? I've never submitted a sitemap and have always pulled up very highly on search engines, I think largely because I have an extremely long list of quality links.

Thanks again, your help is most appreciated

[JamieBrown]

If the robots.txt page in Webmaster Tools says 404 not found then it can't find the robots.txt file on your site. Have you deleted it?

If so, that's a good thing (for the moment) - Google should start indexing all of the URLs on your domain.

No robots.txt generally means "index everything" (as long as you don't have <meta name="robots" content="noindex,nofollow"> on your pages).

[catman]

Thanks again for your help. Yes, we deleted the robots.txt file. I'll most likely get some help in creating a new one once Google starts indexing my pages.