Need help with blocking by user agent in htaccess

Trying to block Mobile/9B206

[grandma genie]

Hello,

I am having some issues with a persistent visitor whose IP keeps changing but the user agent is the same. I am trying to block by using a portion of the user agent, but it is not working. Can anyone see what is wrong? Here is a sample:

RewriteCond %{HTTP_USER_AGENT} Mobile/9B206 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} morfeo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ZmEu [NC]
RewriteRule ^ - [F]

This is the user agent:
Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3

Where am I going wrong with this? Thank you for your help.

-- GG

[wilderness]

Your first issue is in determining the correct forum to seek assistance, this is more appropriate in the SSID forum.

Your thinking these are all the same visitor because of similar UA's and your just wrong!

Nearly all the iPhones have similar UA's.
The IP range could potentially change when they walked across the street to their neighbors house.

Considering that UA's for RewriteCond's lines 2 & 3 are not in the full UA example you provided, makes lines 2 & 3 pretty much useless (unless you use them for something else.

I don't see any reason why you Mobile/9B206 would fail.
Perhaps you have a syntax error somewhere else that is causing these lines to fail?

In addition, all the iPhone lines in my today's logs contained "Mobile/9B206", this if you persist on using this, you may as well use "iPhone" and just deny them all, as it accomplishes the same thing.

Don

[incrediBILL]

Mobile/9B206 is very common in many iPhones, which is why you think 'it' is coming from many IPs because it really is because it's many people from many phones.

Like Don said, might as well just block iPhone itself if you really want to do this.

[grandma genie]

Just assumed that since I was trying to manipulate my htaccess file the question would go in here. And yesterday almost all the visitors had that UA, which is why I thought it was questionable. But today the situation is totally different. Must have just been a fluke. But I appreciate the info about iPhones. I did not realize that particular Mobile number was so common. Considering our increasing mobile society, I don't think I want to block them.

However, since I did try to block it and it didn't work, then there may be a problem with the rest of my htaccess file. I'll check it out. Thank you, Don.

Also I was under the mistaken idea the 9B206 was a model number, but it appears to be a software version that can be applied to a variety of devices.

Considering the number of times it appears in my logs, it must be very popular and there sure are a lot of iPad, iPod and iPhone users out there.

[lucy24]

RewriteRule ^ - [F]

What request would fit this rule? There's a beginning anchor followed by... what, exactly? I'd say .* instead, without anchor. Except I wouldn't really, because you can constrain most rules to requests for pages/directories and then Apache can just cruise on past if an already-approved visitor goes on to ask for css, js, images and whatnot.

Is it a non-human visitor? Mobiles don't get the favicon (my usual #1 test) but they'll get everything else in one quick gulp.

[grandma genie]

Some of the RewriteConds are like this:
RewriteCond %{HTTP_USER_AGENT} ^Sosospider [NC,OR]
and some are not. So will the Rule
RewriteRule ^ - [F]
only work on the entries with the ^? Does that mean I need to separate out the ones without the ^? My list has grown over the years and I just used the initial setup and added to it.

So these would be:
RewriteCond %{HTTP_USER_AGENT} Mobile/9B206 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} morfeo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ZmEu [NC]
RewriteRule .* - [F]

Or does it matter?

I'm not sure if the visitors are bots or humans, but they do grab all files. I get lots of mobile users. I just opened the floodgates today by unblocking all IPs. Just want to see what the difference is in traffic. I did leave the Amazon IPs blocked.

Just based on the activity it appears to be human. But a normal visitor looking for products always puts something in the cart and then checks shipping prices. These visitors never put anything in the cart. They just hit and run.

[wilderness]

gg,
I use the following closing on just about everything, at least for a standard denial.

RewriteRule .* - [F]

Who knows the reason why the iPhone buyers are not making Cart purchases.

I have visitors that simply refuse to use the CONTACT link.
I need something big hand that will reach out of their machine and smack them up side their absurd heads.

[incrediBILL]

But a normal visitor looking for products always puts something in the cart and then checks shipping prices. These visitors never put anything in the cart. They just hit and run.

I shop a lot and rarely put things in carts so you really can't judge by that activity. As a matter of fact, I used to write ecommerce software for about 10 years and the demographics from most stores I've seen show more visitors not putting things in carts than those that do, but it also depends on the store as well.

You might want to check the source of your traffic because different referrers can explain the variation in traffic behavior.

[lucy24]

I have visitors that simply refuse to use the CONTACT link.

They've probably met one too many like the one I did battle with today. After getting yet another mailer-daemon 550 "no such person" message-- yes, really 550, not 511-- after clicking an e-mail link, I spent what felt like several hours (i.e., had to be at least ten minutes) trying to get the Contact link to work on a page within a directory that I think the site administrators meant to delete in 2008 but forgot. First I waited too long and I had to do the captcha all over again, and then it said my text was too long (it took up about 2/3 of THEIR non-scrolling text box) and had to re-capcha. Finally got an auto-response saying they will be in touch with me. I have reason to suspect this is a brazen lie.

---

I think the most recent question was about the ^ anchor. It means "beginning of text". Neither beginning nor ending anchors are necessary unless you specifically need to say "begins with" or "ends with". Or, conversely, if you're saying "does not contain such-and-such at all". Then you need anchors to force the RegEx to evaluate the whole thing, not just the part that fits your group.

But a normal visitor looking for products always puts something in the cart and then checks shipping prices. These visitors never put anything in the cart.

Y'know, I could swear there's a quite recent thread in some forum or other about people shopping with mobile devices. I think there was even some blahblah about different shopping behavior as compared to full-size computers.

(Uhm, what do you call a computer to distinguish it from the small stuff? A normal telephone is apparently now called a "land line", which always makes me think there are an awful lot of people making calls from cruise ships. I generally refer to my iPad as The Toy.)

[wilderness]

They've probably met one too many like the one I did battle with today. After getting yet another mailer-daemon 550 "no such person"

Thanks for the tip.
Perhaps I used an email link on my website (s) in 1999, and when I was using FP to create pages.

I've used a PHP or CGI form, since I don't know when.
Any webmaster whom doesn't test their contact pages is a dunce of the highest degree.

[lucy24]

Urk. Sorry, I was incoherent. Every page has a "contact us" link which leads to a form, but some pages also have an individual e-mail link. That's where I met the mailer-daemon. The actual Contact Us only has a captcha. Oh, yes, and if you do anything wrong-- like filling more than half of the text input area, or including an url to show what you're talking about-- you get taken to a full-fledged error page which I am tolerably certain is not really intended for public consumpiton.

Server Error in '/' Application.
A potentially dangerous Request.Form value was detected from the client (ctl00$MainBodyContent$Body="...mation

<a href="../board/IT...").
Description: Request Validation has detected a potentially dangerous client input value, and processing of the request has been aborted. This value may indicate an attempt to compromise the security of your application, such as a cross-site scripting attack. You can disable request validation by setting validateRequest=false in the Page directive or in the configuration section. However, it is strongly recommended that your application explicitly check all inputs in this case.

... and so on for about 40 more lines. I don't speak .asp so it's just so much Hungarian to me. Possibly the designer forgot the cardinal rule: Don't only make sure it works when it's supposed to. Also make sure it doesn't work when it isn't supposed to. (I found by accident that the site doesn't even have a custom 403 page, although the host apparently expects them to. Sheesh.)

I've got naked e-mail links on my own site. And almost the same on the art studio's site, just disguised as an html form. But then, I'm not the top widget supplier for the Atlantic seaboard, so I'm not much plagued with spam.

Besides, "contact us" links make me anxious as a user. You never know what information they're going to demand or how many things you're allowed to click before there's no going back.

[grandma genie]

Well, at least I know my Contact Us page works.

I'm going to use Don's example for my htaccess file and see what happens. RewriteRule .* - [F]

Meanwhile, since removing most of my IP blocks, I am getting more traffic, but some I'll need to block again. Sort of like starting from scratch again. I'll see how it goes.

Thank you for all your help.

[lucy24]

Sort of like starting from scratch again.

This can be useful. Sometimes you'll get a robot that pesters you every day for a week, so you block them-- and then when you check logs months later, you'll find that they never came back again anyway. All those nanoseconds of reading an htaccess file do add up.