My robots.txt has Googlebot allowed (apart from a few directories/files) but Googlebot-Image/1.0 disallowed from all directories. The reason behind this is that I do not wish my images indexed on Google search as they are of childrens orgnization and as such the members privacy is paramount.
I noticed in my cPanel logs the following:
[Fri Oct 15 10:08:49 2010] [error] [client 66.249.65.14] client denied by server configuration: /home/x/public_html/directory/photo.jpg
[Fri Oct 15 10:08:49 2010] [error] [client 66.249.65.14] client denied by server configuration: /home/x/public_html/robots.txt
If I am reading the above correctly, not only was Google denied from indexing the photo itself but the robots.txt also.
Is the above IP Googlebot-Image or just Googlebot?
How can I ensure Googlebot(-Image or otherwise) is permitted to read my robots.txt?
Would an entry in my robots.txt reading 'Disallow: /r' also deny access to my robots.txt file if it is located in the root directory?
Many thanks in advance.
