Forum Moderators: open
robots.txt? Yes, but inconsistent.
Fake referer? Yes, and inconsistent: Your home URL or a page on your site
Same IP+UA seen on two disparate sites. IP registered to Yellowpages.com.
Link in UA 404s and shows a "CASINO BONUS OFFER" image w/ gaming and betting links. Home page similar. (Oh, Yellow Pages -- what's up with that?)
Slightly earlier, slightly different heritrix version seen last month --
Mozilla/5.0 (compatible; heritrix/1.14.2 yptrino +http://www.buddybuzz.net/yptrino)
-- hailing from amazonaws.com. (See "amazonaws.com plays host to wide variety of bad bots [webmasterworld.com]." Message #3911669)
Same version also seen last May:
ec2-174-129-224-44.compute-1.amazonaws.com
Mozilla/5.0 (compatible; heritrix/1.14.2 yptrino +http://www.buddybuzz.net/yptrino)
robots.txt? Yes
Fake referer? No
Older thread: [webmasterworld.com...]
I even find their Yellow SPAM on the doorsteps of the residents in my building. Had to call them to send their truck back to remove 'em.