homepage Welcome to WebmasterWorld Guest from 54.211.68.132
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Beware HTML in abUser Agent Strings
filter HTTP requests and logs like any other form of input data
incrediBILL




msg:4475729
 12:59 am on Jul 15, 2012 (gmt 0)

Never saw anyone try this before and I've seen just about everything including javascript in user agents trying to redirect to malware sites.

188.190.124.66

<br><center><h1>Pilferer</h1></center><br><center><h2><a href="http://example.com/">[Toolly Robot 0.1]</a></h2></center>


FYI, I replaced the domain with example.com because this one is not safe to surf and I'm not promoting it here just in case someone isn't careful and got hammered by accident.

Beware as user agent strings, referrers and even the requested pages are a potential vulnerability, just like cross-site scripting, which could be used to attack to your MySQL queries and reports if not properly filtered.

Best case, as in the above example, the code just messes up the report and displays the user agent in a big font in the middle of the screen with a link to their site.

Worst case, it contains a javascript redirect and or attempts at MySQL attacks.

People tend to filter malicious input from being passed to their web pages but ignore the fact that this data can also logged in it's raw form and then processed later by server stats programs that don't always filter that data properly to protect webmasters from simplistic browser or SQL attacks. Once I noticed this stuff happening I stopped all server side log analysis programs as I can't be sure they are secure and won't let something through that puts my server or browser at risk.

 

wilderness




msg:4475756
 2:19 am on Jul 15, 2012 (gmt 0)

Bill,
I had one of these a month or two ago and simply shrugged it off as an errant spammer. Failing to make a saved notation.
Not sure I'm willing to dig back through the logs attempting to locate it.

incrediBILL




msg:4475758
 2:32 am on Jul 15, 2012 (gmt 0)

It's more than just that, some are spamming, some are attacking.

Not all of the sites are just spam links, some are malicious.

FWIW, I'm getting tons of this stuff lately on a couple of sites. All reports I wrote have now been verified to be HTML sanitized and I shut down any log analysis and reports that I didn't write for security.

keyplyr




msg:4475762
 2:50 am on Jul 15, 2012 (gmt 0)

My reports are hot, but I never follow any of the links. If I want to investigate, I either load the link into a browser manually with noscript & adblock, JS off and redirects off or I use a 3rd party web-based HTML parser tool.

incrediBILL




msg:4475795
 5:44 am on Jul 15, 2012 (gmt 0)

Does that mean you have javascript disabled for your own site for those report pages?

If not, they can easily hijack your page with a javascript redirect which is quite annoying.

I found out ages ago that so many email submission pages, shopping carts, and all sorts of common stuff including back end admin tools are quite susceptible to simple javascript redirects so I wasn't exactly surprised when I saw people trying to take advantage of log files reports.
.

keyplyr




msg:4475801
 5:56 am on Jul 15, 2012 (gmt 0)



If not, they can easily hijack your page with a javascript redirect which is quite annoying.

Not possible on my server.

Also, I hand code input forms, don't use anything off the web or out of a box. I build in a few security measures.

MickeyRoush




msg:4476061
 2:07 pm on Jul 16, 2012 (gmt 0)

My reports are hot, but I never follow any of the links. If I want to investigate, I either load the link into a browser manually with noscript & adblock, JS off and redirects off or I use a 3rd party web-based HTML parser tool.


You may want to include RequestPolicy also.

https://www.requestpolicy.com/
https://www.requestpolicy.com/faq.html#faq-noscript
https://addons.mozilla.org/ja/firefox/addon/requestpolicy/

keyplyr




msg:4476164
 6:41 pm on Jul 16, 2012 (gmt 0)

@MickeyRoush - yeah good security tool as well. I also use DoNotTrack Plus, opt out where possible... ad infinitum.

motorhaven




msg:4476220
 10:00 pm on Jul 16, 2012 (gmt 0)

Things like this are why I'm glad I do my reports using an extensive collection of shell scripts I've written over the years.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved