Page is a not externally linkable
londrum - 8:16 pm on Nov 18, 2010 (gmt 0)
Q: I want to block my images from being indexed, but I’m happy with them appearing on a preview image; how can I juggle the two?
A: In order for images to be embedded in previews, it is important that they are not disallowed by your robots.txt file. In order to block crawlable images from being indexed, you can use the "noindex" x-robots-tag HTTP header element.
if we remove the block in robots.txt, doesnt that mean that all the thousands and thousands of images we have on our sites will now get downloaded by all the bots? because they wont be able to tell its noindex until they grab it. or does it not work like that?