Welcome to WebmasterWorld Guest from 22.214.171.124
Why is Google indexing my entire web server?
The consensus is that you need to block your site on the server if it's under development or if you don't want it indexed. Lack of inbound links is not enough, as publicly available server logs are likely to get spidered by Googlebot. Use password protection or the no-index robots meta tag on pages you want to block.
To block all (good) bots just make a robots.txt file and enter: