For years I have had DigExt in the banned robots list in .htaccess but have decided to remove it.
Just now, a quick search has turned up several posts indicating it was questionable, but nothing I found seemed definitive.
However -
Consider today this UA requesting a legitimate file:
GET <mysite/myfile> HTTP/1.1" 403 - "<google URL>url?sa=t&rct=j&q=<innoculous query of six word length>source=web&cd=3&sqi=2&ved=0CEAQFjAC&url=<mysite/myfile>&ei=Bn5sT64YycmJAsPZsK0F&usg=AFQjCNGHBy6ychCw7_b0fMYPCqD6zyrW8A" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; DigExt; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2; ShopperReports 3.0.497.0; SRS_IT_E8790571B4765C503EAC98)"
I've replaced the exact query. A substitute might be, say "stories from Africa about conservation problems" - a rather authentic looking real-person query. About 90 secs later the follow-up query was, say, "stories about conservation problems" ... which seems quite human. I may have banned an innocent request. That was the only instances of DigExt today.
But, hold on -
Looking at yesterday, however, I found:
GET forum.<mysite>.com/" "Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt)"
The forum request came 17 times yesterday and 12 times so far today, with various other assorted (none DigExt) UAs - all from different DIRECT IPs - not through the threat-filter proxy as any request made by domain name
I have no forum. Never have. That's a real robot. It's banned anyway for its guessing.
My conclusion is that DigExt was included at random as one of the UAs used by a robot, but that DigExt also appears in apparently authentic requests.
I'm going to take if off my robot list, and let the robots get trapped by more specifically malicious behavior that their UA.
The reason for my post is not so much by was of asking for comments, but more to provide the innocent-looking appearance of DigExt in my limited two-day look at it. This comment may help others wondering what it is.
