Forum Moderators: goodroi

Message Too Old, No Replies

Removed 'deny all' robots.txt but no change

         

kapow

5:01 pm on Oct 22, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



A site I manage has had a robots.txt with 'deny all' for about 3 years. 4 months ago the site owner changed his mind and asked me to get the site onto search engines. I removed the robots.txt.

The site still has PR0 on the Google toolbar, other sites I launched 4 months ago are showing some Google PR.

How can I be sure Google knows to spider the site. It has no robots.txt now, and it has several quality links pointing to it. I also manually submitted to Google with a note to explain I just removed the robots.txt.

Span

11:03 pm on Oct 23, 2004 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Just a thought. Maybe it's better to let the bots know the domain is allowed to be spidered now. Bots have some sort of memory: [google.com ]. So I'd put a robots.txt in there with this:

User-agent: *
Disallow: