|Weird 404s in stats|
real url with long jumbled strings
| 12:17 pm on Apr 10, 2013 (gmt 0)|
Hope this is the right forum. I noticed an increase in visits and looked at the visit details and found a lot of 404s that showed a real url appended by a long string of uniquely jumbled letters. Each comes from a different visitor, mostly outside of US, all of which are using "Other Agent (unknown platform)" as the type of browser, and each is to the same url except for the appended jumbled string. Is this someone trying to use a comment spambot on a page without comments? It's a static html site.
| 6:20 pm on Apr 10, 2013 (gmt 0)|
File under: botnet run by someone who's not too bright. (Isn't the whole point to keep the original UA so you come in looking like a human?) Personally I'd just block the UA and think no more about it ;)
|a real url appended by a long string of uniquely jumbled letters |
Do you mean things like
If requests in this form become troublesome, you can also add a line to htaccess that says something like
RewriteRule \.html. - [F]
or, in the alternative,
RewriteRule ^([^.]+\.html). http://www.example.com/$1 [R=301,L]
depending on whether you want to lock them out with no further discussion, or redirect them to the proper URL. With robots it probably makes no difference; when they see a 301 they generally go off sulking and don't come back. A redirect uses fewer server resources as you don't have to go through the motions of offering a 403 page instead.
| 12:41 pm on Apr 12, 2013 (gmt 0)|
I mean things like www.example.com/nameofrealfile.html&sa=U&ei=905lUdSPFqGo4AT1xoBw&ved=0CBgQFjAA&usg=AFQjCNHyUazBsXhWP5Yn5ZIrHEWRkQGbaA
They come from all diffeent ip addresses, so I can't block based on that. They all start with the same real file name but each visit has a different string of random letters and symbols appended to it. These accounted for 20% of individual visits for a few days.
| 12:49 pm on Apr 12, 2013 (gmt 0)|
These are technically not 404 errors, as the server would deliver a '200 ok' since they are showing up,as real visits in your stats. Can you see a referrer? Google tends to add something like this when searches are coming from someone logged in.
| 8:42 pm on Apr 12, 2013 (gmt 0)|
|These are technically not 404 errors, as the server would deliver a '200 ok' since they are showing up,as real visits in your stats. |
That's the default behavior. But if you turn off AcceptPathInfo, they will become 404s.
The OP said
|all of which are using "Other Agent (unknown platform)" as the type of browser |
I assumed you were talking about a literal string here. You can block by User-Agent. The easiest way is to use mod_setenvif in conjunction with mod_authz-whatsit:
BrowserMatch "Other Agent" keep_out
(listing all the browsers you unconditionally dislike, one per line)
Deny from env=keep_out
in the same place you list your IP lockouts.
| 12:17 pm on Apr 13, 2013 (gmt 0)|
Thanks, lucy24. I will give that a try.