Forum Moderators: goodroi

Message Too Old, No Replies

User Agent - Image Search

Trying to find the user agent for ask

         

Jamie_uk

2:10 pm on Apr 10, 2006 (gmt 0)

10+ Year Member



I'm trying to find the user agent i can use to stop "ask" indexing images on my site, the site is huge and all images are simply not in one directory therefore i was hoping to take advantage of something similar to google and yahoo's user agenets:

User-agent: Yahoo-MMCrawler
User-agent: Googlebot-Images

Do ask have one?

DanA

2:13 pm on Apr 10, 2006 (gmt 0)

10+ Year Member



It may be this one :
User-agent: Teoma

Ask User agent being
Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)

Pfui

3:25 pm on Apr 10, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



Note: Google's image crawler is 'singular':

User-agent: Googlebot-Image

1.) Here's the "Ask Web Crawler FAQ [sp.ask.com]" with more details and ayep, it's:

User-agent: Teoma

I don't see a separate crawler ID for images. I do see the following UA from similar IPs, asking for robots.txt:

"Mozilla/2.0 (compatible; Ask Jeeves/Teoma; +http://sp.ask.com/docs/about/tech_crawling.html)"

crawler100.ask.com
egspd42059.teoma.com
egspd42452.ask.com

2.) Here are real visitors with Ask/Teoma-related UA and referers. I was the first referer, testing a saved link to one of my own images to "mystuff.ask.com":

http:// mystuff.ask.com/mysearch/FullImage?
http:// mamma30.mamma.com/Search?evid=CE0070178928&eng=Teoma&cb=MammaRS&dest=
http:// www.ask.com/web?q=
http:// uk.ask.com/web?q=

"Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; TeomaBar 2.01; TheFreeDictionary.com)"

3.) Ask (formerly AskJeeves) has been sneaking into my site using the same IP and UA for months and months -- and not asking for robots.txt. Also doesn't get any images, tries to get just one (different) file at a time. FWIW:

65.214.39.180
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)
04/06 12:26:47 /dir1/file1.html

65.214.39.180
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.1.4322)
04/04 19:09:11 /dir2/file2.html

Working backwards, excerpted:

65.214.39.180 - - [01/Mar/2006:06:54:44 -0800]
65.214.39.180 - - [05/Mar/2006:19:26:53 -0800]
65.214.39.180 - - [07/Mar/2006:20:33:48 -0800]
65.214.39.180 - - [16/Mar/2006:14:35:59 -0800]
65.214.39.180 - - [21/Mar/2006:07:52:09 -0800]
65.214.39.180 - - [23/Mar/2006:07:35:47 -0800]
65.214.39.180 - - [31/Mar/2006:19:10:17 -0800]
(Etc.)
65.214.39.180 - - [25/Dec/2005:19:44:53 -0800]
65.214.39.180 - - [09/Dec/2005:19:20:17 -0800]

WHOIS excerpted from dnsstuff:

CustName: AskJeeves, Inc.
Address: 5858 Horton Street
City: Emeryville
StateProv: CA
PostalCode: 94608

NetRange: 65.214.36.0 - 65.214.39.255

(Probably too much info, sorry, but there you go:)