| Welcome to WebmasterWorld Guest from 184.108.40.206 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Is there a Fast robot for images?|
Looking for a name to use in robots.txt
| 5:53 pm on Jul 9, 2002 (gmt 0)|
Does Fast have a special robot that requests images?
This is a quick question because I am only asking about a robot that can be disallowed in robots.txt
| 6:45 am on Jul 10, 2002 (gmt 0)|
Hi, this is Stig from FAST.
We do have a crawler for images and other formats, it is using the agent string "FAST/WebCrawler-3.x Multimedia".
| 7:41 am on Jul 10, 2002 (gmt 0)|
Thank you. I think this is what I am looking for. But will this work?
User-agent: FAST/WebCrawler-3.x Multimedia
Or do I have to write this some other way?
| 8:16 am on Jul 10, 2002 (gmt 0)|
I had not noticed you around here before. Welcome.
| 9:46 pm on Jul 10, 2002 (gmt 0)|
Misosoph: yes, that robot.txt file should work. Please note that your robots.txt file changes won't apply until we re-fetch that file. This should not take longer than three to four days.
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved