homepage Welcome to WebmasterWorld Guest from 54.234.0.85
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
Robot.txt file deleted by accident what now?
Hoppo




msg:1525394
 10:48 pm on Dec 12, 2004 (gmt 0)

Sorry everyone, to bore with a repetetive question.

I have been handed a site(with little experience myself) changed a few things including the page title on index page, (using frontpage 2003) added a few keywords to site content pages......then uploaded new site contents then dead..no listings anymore in google... whereas there used to be many! Gulp!

I have waited for over five weeks in the hope that my new revamped (or cocked up) site would be re-indexed but nothing. In desperation I have removed robots.txt file as it read as follows...

# for use by all robots
User-Agent: *
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /scripts/
Disallow: /styles/
Disallow: /Thumbs.db

And I think I may have totally misundertood everything here. I thought this meant that none of my site would get read?

Did I do wrong and should I put this back, please help going mad day by day.

Hoppo.

 

treeline




msg:1525395
 11:15 pm on Dec 12, 2004 (gmt 0)

If you have no robots.txt then robots will feel free to look at anything they want.

Your previous robots.txt excluded robots from much of your website. If all of your pages were in the excluded directories, they may have concluded you didn't like company and are politely keeping their distance.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved