Yahoo-MMCrawler

what is this?

[jackson]

Got "smacked" with this one this week as in:

205.188.116.*:Yahoo-MMCrawler/3.x :Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.4

Had it crawling through folders that had been disallowed in my robots.txt which is rather "unYahoo-like".

Looks like a "spoofed" header. Doing an whois on the IP address throws up AOL. Didn't know that Yahoo was using Apple gear to do their work. Can anyone throw some more light on this item?

[edited by: volatilegx at 6:00 pm (utc) on Aug. 12, 2004]
[edit reason] obscured IP address [/edit]

[wilderness]

I've been getting some blank refer and ua's from AOL IP's for some time.
My visitors may not be related however it seems pretty coincedental?

Yesterday I had a AOL IP with blank refer and ua make approx 20 consecutive requests for some PDF's I have online. The result was a matching number of 403's.
I made an inquiry to some AOL users a while back if perhaps there was an AOL patch or upgrade which change the refer and ua's to blank. Nobody knew :-((

Thanks for the heads up.

Don

[fiestagirl]

Discussion about the multi-media spider here:

[webmasterworld.com...]