homepage Welcome to WebmasterWorld Guest from 54.166.122.65
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

    
Is this a valid user agent? and how do I block it?
Is this a valid user agent? and how do I block it?
spiritualseo

5+ Year Member



 
Msg#: 4404758 posted 2:52 pm on Jan 7, 2012 (gmt 0)

Hey guys, is this a valid user agent?

Mozilla/5.0 (X11; U; Linux i686; it-IT; rv:1.9.0.2) Gecko/2008092313 Ubuntu/9.25 (jaunty) Firefox/3.8

Have been getting a lot of referral spam mainly from #*$! websites from the above user agent. If this is invalid, any help on how I can block this in htaccess? Thank you!

 

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4404758 posted 7:17 pm on Jan 7, 2012 (gmt 0)

The UA is common. Block by IP address.

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4404758 posted 10:41 pm on Jan 7, 2012 (gmt 0)

Jaunty is out of date now - I'm running 10.04 Lucid, which is also being slowly phased out.

I would be wary of the term Ubuntu/9.25. Ubuntu versions are either n.04 or n.10 - Jaunty is 9.04. A search for Ubuntu/9.25 does not produce many results and at least one of those says "forge your UA".

On Lucid I'm currently running firefox 9.1. Ubuntu seemed to be pushing 3.6 until a few months ago - I went straight from 3.6 to 7.something. Can't say I've heard of 3.8 for Ubuntu.

tangor

WebmasterWorld Senior Member tangor us a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month



 
Msg#: 4404758 posted 10:56 pm on Jan 7, 2012 (gmt 0)

Sometimes we address the referer, too. Particularly with pron sites... there's aren't that many "words" to deal with. :)

spiritualseo

5+ Year Member



 
Msg#: 4404758 posted 7:48 pm on Jan 8, 2012 (gmt 0)

So would it be safe to block Ubuntu/9.25? Will something like this work:

RewriteCond %{HTTP_USER_AGENT} Ubuntu\ 9.25

dstiles

WebmasterWorld Senior Member dstiles us a WebmasterWorld Top Contributor of All Time 5+ Year Member



 
Msg#: 4404758 posted 9:26 pm on Jan 8, 2012 (gmt 0)

I would check previous logs and do a bit of research to be sure about blocking it.

As to the rewrite - can't help there, I'm afraid: I don't run htaccess.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4404758 posted 10:36 pm on Jan 8, 2012 (gmt 0)

So would it be safe to block Ubuntu/9.25? Will something like this work:

RewriteCond %{HTTP_USER_AGENT} Ubuntu\ 9.25


You've correctly escaped the literal space, but you also need to escape the literal period. Unless you're expecting visits from "Ubuntu 9425" or "Ubuntu 9a25" or even "Ubuntu 9 25" and want to block those too. Which, come to think of it, wouldn't do any harm. But an unescaped period can lead to unwanted consequences, so it's best to stay in the habit.

And then, ahem, you need a Rule to go with the Cond ;)

spiritualseo

5+ Year Member



 
Msg#: 4404758 posted 5:35 pm on Jan 9, 2012 (gmt 0)

I did check the previous months logs and nearly all traffic from this agent is spam. Successfully blocked this using the following:

RewriteCond %{HTTP_USER_AGENT} Ubuntu\/9\.25 [NC]
RewriteRule .* - [F,L]

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4404758 posted 6:44 pm on Jan 9, 2012 (gmt 0)

Just a FYI

RewriteRule .* - [F,L]


"F" means final and "L" means last, so in effect you are saying the same thing twice. Won't hurt, but a more succinct way of writing this would be:

RewriteRule .* - [F]

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4404758 posted 12:25 am on Jan 10, 2012 (gmt 0)

[F] doesn't mean Final, it means Forbidden (think "Fail")-- like Deny from. It's one of a handful of flags that carries an implied [L]. But it's a good habit to include [L] with each separate RewriteRule unless you've got a specific reason to exclude it, so you never leave it out by accident.

In Apache you don't escape / slashes. You're thinking of javascript. Or was that a typo for the same escaped space as before?

keyplyr

WebmasterWorld Senior Member keyplyr us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4404758 posted 2:04 am on Jan 10, 2012 (gmt 0)

[F] doesn't mean Final, it means Forbidden
Oh yeah, my bad. Since I only write regex that forbids something I guess I see it as final if there are no more rules. However, I never use both F and L and consider it redundant.

In Apache you don't escape / slashes. You're thinking of javascript. Or was that a typo for the same escaped space as before?

No need to escape forward URL slashes in JS either.

lucy24

WebmasterWorld Senior Member lucy24 us a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



 
Msg#: 4404758 posted 4:59 am on Jan 10, 2012 (gmt 0)

:) Except in Regular Expressions, because the slashes are what demarcates them.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved