Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Froogle wants my images!



12:47 pm on Apr 6, 2004 (gmt 0)

Inactive Member
Account Expired


I've tried to add a store to froogle, and have recieved an email saying that my robots.txt doesn't allow access to my images folder. Fair enough.

My current rule is:

User-agent: *
Disallow: /css
Disallow: /javascript
Disallow: /admin
Disallow: /images
Disallow: /downloads

What I want to know is, if I add

User-agent: googlebot
Disallow: /css
Disallow: /javascript
Disallow: /admin
Disallow: /downloads

Will that work, or will googlebot 'inherit' the Disallow: /images rule from 'User-agent: *'? Alternatively, is their a better way of doing it?


5:41 am on Apr 14, 2004 (gmt 0)

Preferred Member

10+ Year Member

joined:Mar 31, 2004
votes: 0

Why don't you allow the froogle agent access to the images folder and block all else.
5:59 am on Apr 14, 2004 (gmt 0)

Senior Member

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Mar 31, 2002
votes: 0

Put the more specific record first. Googlebot, and then "*".

Most if not all robots accept the first record that matches their user-agent string.