| Welcome to WebmasterWorld Guest from 22.214.171.124 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Is there a Fast robot for images?|
Looking for a name to use in robots.txt
Does Fast have a special robot that requests images?
This is a quick question because I am only asking about a robot that can be disallowed in robots.txt
Hi, this is Stig from FAST.
We do have a crawler for images and other formats, it is using the agent string "FAST/WebCrawler-3.x Multimedia".
Thank you. I think this is what I am looking for. But will this work?
User-agent: FAST/WebCrawler-3.x Multimedia
Or do I have to write this some other way?
I had not noticed you around here before. Welcome.
Misosoph: yes, that robot.txt file should work. Please note that your robots.txt file changes won't apply until we re-fetch that file. This should not take longer than three to four days.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved