Forum Moderators: goodroi
User-agent: *
Disallow:
Ie is there some latency in the way spiders remember the robots.txt even after it has been deleted - if you don't specifically countermand the robots.txt with a new version?
(in particular I am interested in google)