Msg#: 4426868 posted 8:38 am on Mar 9, 2012 (gmt 0)
My robots.txt previously allowed Google to index almost everything in my site except my image directory, whereas Google Web Preview had similar permissions but was permitted to index the image directory. The problem I'm finding is that despite allowing GWP to index my images, the actual web preview results while using Google search showed my pages minus the images. Clearly this isn't needed to be addressed so I've now reluctantly permitted Google to index the images directory via robots.txt
Should this change fix the GWP results or is there anything else I need to change? Is there a was of permitting GWP to use my sites images without letting Google search index them? (I'd refer them not to show up in 'image' search results).
Msg#: 4426868 posted 11:49 am on Mar 9, 2012 (gmt 0)
Tangential answer: GWP is not a robot. You and I may think it is, but g### knows better. Preview doesn't even look at robots.txt, and it doesn't index. It just fetches a page in response to human requests based on ordinary googlebot activities.
When you say Google, do you include Googlebot-Image? It seems to think of itself as a separate robot, though occasionally the regular googlebot will pick up images. (Sometimes with a referer, just to confuse us. Recent discussion around here somewhere.)
Another place to look is your hotlinking codes. If it only goes by referer you're fine, because GWP will list your own page as referer for its images. But if you have anything about UA or IP, make sure GWP can get in.
Edit: I'm pretty sure I've checked this before but I just went back to double-check. If you Preview a page that uses roboted-out images, Preview will show those images.
Msg#: 4426868 posted 12:15 pm on Mar 9, 2012 (gmt 0)
Thanks Lucy. I'll have a good look around then to check what I may have set to stop GWP loading my images. I've just tried again and they're still not loading despite my robots currently allowing everything access to my images folder. Going by your edit though, even that shouldn't be stopping them loading.
Edit: I think it may have been due to hotlink protection in my images/.htaccess I've added an exception for Google, lets see if that makes a difference.