Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

Froogle wants my images!



12:47 pm on Apr 6, 2004 (gmt 0)

I've tried to add a store to froogle, and have recieved an email saying that my robots.txt doesn't allow access to my images folder. Fair enough.

My current rule is:

User-agent: *
Disallow: /css
Disallow: /javascript
Disallow: /admin
Disallow: /images
Disallow: /downloads

What I want to know is, if I add

User-agent: googlebot
Disallow: /css
Disallow: /javascript
Disallow: /admin
Disallow: /downloads

Will that work, or will googlebot 'inherit' the Disallow: /images rule from 'User-agent: *'? Alternatively, is their a better way of doing it?


Import Export

5:41 am on Apr 14, 2004 (gmt 0)

10+ Year Member

Why don't you allow the froogle agent access to the images folder and block all else.


5:59 am on Apr 14, 2004 (gmt 0)

WebmasterWorld Senior Member jdmorgan is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Put the more specific record first. Googlebot, and then "*".

Most if not all robots accept the first record that matches their user-agent string.



Featured Threads

Hot Threads This Week

Hot Threads This Month