Fetch API Request

Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Fetch API Request

[jamesyap]

I think most of you has seen this user agent in your log files. I have seen it for months and they are hungry little spider which usually follow all the links in my site.

There are post about it here in webmasterworld but identity are 'maybe' instead of 'for sure'.

Here is a page with some info [psychedelix.com...]

That's 'maybe' too. It might possibly be a 'ZIP' tool which zip your entire site including all your IMAGES. Lets find out by reviewing more information from the log file.

62.252.64.8 (inktomi5-ltn.server.ntl.com)
NTL.com is an ISP website! it look it it is from a live broadband user!

196.40.25.163 (not resolvable, not pingable)

Both grab entire site with all images. Are they some kind of ZIP tools and do you think they follow the robots.txt file?

[wilderness]

James,
Don't recall what or who they are at the moment except a PITA :)

This resolves their insistence though :)
SetEnvIf User-Agent Request$ keep_out
SetEnvIf User-Agent Request.5)$ keep_out

Don

[jamesyap]

Do you mean I should place the 2 lines in .htaccess? But my server has frontpage extention installed.

[wilderness]

James,
Somebody who modifies their htaccess while "publishing" Front Page will hopefully advise you. We have one such participant here although I'm embarrased I don't recall his name.

My pages are for the most part created with FP as a text editor. I've never used nor will I "publish" or any of the FP bot componets.
I'm not being critical of your preference.
Just that my methods have been successful thus far and why rock the boat :)

Don

[jdMorgan]

You might want to try this unanchored one-liner to catch all of the variants:

SetEnvIf User-Agent "Fetch\ API\ Request" keep_out

It might even work without the quotes - I haven't tried it:

SetEnvIf User-Agent Fetch\ API\ Request keep_out

Jim