homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

what are the names of the image bots?
I can't ban them if I don't know their names

 2:38 am on Dec 25, 2004 (gmt 0)

I would prefer that my images not be entered into the image search engines. I've banned Google's image bot, but I'm unable to do the same for Yahoo's or Ask.com's, because I don't know the names of their spiders.

A link to a listing of current bot names would be great.

Thank you.




 11:04 pm on Dec 27, 2004 (gmt 0)

User-agent: googlebot-image
User-agent: psbot (picsearch)
User-agent: Yahoo-MMCrawler

Don't know about AJ.


 11:53 pm on Dec 27, 2004 (gmt 0)

Is there any point in not Disallowing the images folder? I thought the bots were interested only in text.


 12:25 am on Dec 28, 2004 (gmt 0)

The image-bots are interested in images. That's sorta the point of image-bots.



 3:17 am on Dec 28, 2004 (gmt 0)

psbot appears to be ignoring my robot.txt file. Has anyone out there noticed this?

I'll double check everything and report back in a couple of days.


 11:49 am on Dec 28, 2004 (gmt 0)

stapel, bear with me, I am still learning about bots. What is the purpose of an images bot - and what do they latch on to? As I see it, the image has a file name only, which doesn't mean much. Do the bots link the file name to the alt text in the HTML file? And anyway, why deny bots access? If images are confidential, there does not seem much point having them on a web site (other than a secure one).

Not tyring to be argumentative, just trying to learn.


 2:10 pm on Dec 28, 2004 (gmt 0)

The purpose of an image-bot is to harvest images. They latch onto images, as well as surrounding text which may be helpful in returning hits to return in image searches.

Images don't have to be confidential for the creator not to want them included in a "go ahead and nick any of these images" search engine.



 5:17 pm on Jan 2, 2005 (gmt 0)


OK, thanks.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved