homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
Forum Library, Charter, Moderator: open

Deprecated - Altavista, Alltheweb.com Forum

Is there a Fast robot for images?
Looking for a name to use in robots.txt

 5:53 pm on Jul 9, 2002 (gmt 0)

Does Fast have a special robot that requests images?

This is a quick question because I am only asking about a robot that can be disallowed in robots.txt




 6:45 am on Jul 10, 2002 (gmt 0)

Hi, this is Stig from FAST.

We do have a crawler for images and other formats, it is using the agent string "FAST/WebCrawler-3.x Multimedia".

- Stig


 7:41 am on Jul 10, 2002 (gmt 0)

Thank you. I think this is what I am looking for. But will this work?

User-agent: FAST/WebCrawler-3.x Multimedia
Disallow: /

Or do I have to write this some other way?

Thanks again.


 8:16 am on Jul 10, 2002 (gmt 0)

Hi Stig

I had not noticed you around here before. Welcome.


 9:46 pm on Jul 10, 2002 (gmt 0)

Misosoph: yes, that robot.txt file should work. Please note that your robots.txt file changes won't apply until we re-fetch that file. This should not take longer than three to four days.

- Stig

Global Options:
 top home search open messages active posts  

Home / Forums Index / Yahoo / Deprecated - Altavista, Alltheweb.com
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved