homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
google images ignoring disallow
apollo




msg:1525817
 2:32 am on Mar 25, 2004 (gmt 0)

I have this text in the robots.txt file:

*******************************
User-Agent: Googlebot-Image
Disallow: /
*******************************

This is the only example I could find on the net regarding disallowing google images. In validation it gives this warning:

********************************
1 warning Field names of robots.txt maybe case insensitive, but do capitalize field names to account for challenged robots. (eg: User-agent)
User-Agent: Googlebot-Image
******************************

My images are now appearing in google image search. Is my robots.txt syntax correct?

When I set up this file google was the only engine crawling images but now there are plenty but I still only ban google. Is there anyway to ban all image crawlers?

Here is my full robots.txt:

User-Agent: Googlebot-Image
Disallow: /

User-Agent: ia_archiver
Disallow: /

Thanks

 

pixel_juice




msg:1525818
 10:43 pm on Mar 25, 2004 (gmt 0)

The validation error isn't very clear, but you need to use a lower case 'a' in User-agent I think ;)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved