Forum Moderators: phranque
If you use HTTP authentication spiders all receive a 401 or 403 response since they don't supply login credentials, and search engines don't list pages with these status codes.
My problem is that I need it to be live and not on my local machines - aside me it is also accessed by my CSS person and graphic partner
If there wil only be a handful of people working with the site, you can make the site accessible for those IPs. In Apache .htaccess this would be something like:
Order deny,allow
Deny from all
Allow from 111.111.111.111 222.222.222.222