Welcome to WebmasterWorld Guest from 54.162.239.134

Forum Moderators: goodroi

Message Too Old, No Replies

Robot.txt file deleted by accident what now?

     
10:48 pm on Dec 12, 2004 (gmt 0)

10+ Year Member



Sorry everyone, to bore with a repetetive question.

I have been handed a site(with little experience myself) changed a few things including the page title on index page, (using frontpage 2003) added a few keywords to site content pages......then uploaded new site contents then dead..no listings anymore in google... whereas there used to be many! Gulp!

I have waited for over five weeks in the hope that my new revamped (or cocked up) site would be re-indexed but nothing. In desperation I have removed robots.txt file as it read as follows...

# for use by all robots
User-Agent: *
Disallow: /cgi-bin/
Disallow: /images/
Disallow: /scripts/
Disallow: /styles/
Disallow: /Thumbs.db

And I think I may have totally misundertood everything here. I thought this meant that none of my site would get read?

Did I do wrong and should I put this back, please help going mad day by day.

Hoppo.

11:15 pm on Dec 12, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If you have no robots.txt then robots will feel free to look at anything they want.

Your previous robots.txt excluded robots from much of your website. If all of your pages were in the excluded directories, they may have concluded you didn't like company and are politely keeping their distance.

 

Featured Threads

Hot Threads This Week

Hot Threads This Month