Page is a not externally linkable
Rumbas - 2:41 pm on Nov 20, 2003 (gmt 0)
In a perfect world, yes. However you should always put a robots on your server, even if it's empty. All good spiders will ask for it and obey it too.
>if i dont create an robots.txt, all the spiders will crawl my site?