Welcome to WebmasterWorld Guest from 188.8.131.52
Forum Moderators: goodroi
But the server is returning 403 errors for everything for the 404s and particularily the robots.txt file...
Do search engines care if non existent robots.txt file
is showing them 403 errors instead of the correct 404 errors?
I apologize for putting this question in different forums but it was really worrying me that the site might not get listed because of the 403 errors... I uploaded a blank robots.txt file just in case.
> But the server is returning 403.
... And that response is incorrect and you should find out why. Just because Google has a work-around, doesn't mean that others do. And maybe their work-around will be accidentally removed some day... Unlikely, but do you want to bet your business?
I strongly suggest you find out why your server is incorrectly returning a 403 server response and fix it.
If your site does not meet HTTP specifications, it is a disaster waiting to happen. 403 has a meaning and 404 has a meaning, and they are definitely not interchangeable! Also, put up at least a blank robots.txt file and avoid all those 404's in your logs!
"This is because the folder is control. If you look for the same file in an uncontrolled folder you will get the 404 error. www.mydomain.com/images/robots.txt "
And I completely agree as to an apple is an apple and an orange is an orange in the configuration of the server and the errors being generated.
The operating system is: Linux 2.2.19-6.2.11
The web server is: Apache/1.3.9 (Unix) (Red Hat/Linux) PHP/4.3.1