Amazon bot and friends?

Possible botnet visit?

[grandma genie]

Just for your info, I had a visit from an Irish IP 89.191.34.nnn behaving in a normal fashion. Checked out one page, text and images, put an item in the shopping cart and abruptly left. Very next hit was this looking at the same page as the Irish visitor:

184.73.103.nnn - - [22/Oct/2012:06:25:47 -0400] "GET /directory/example.html HTTP/1.1" 403 - "-" "bitlybot"

This is an Amazon IP, so was blocked in htaccess.

Then the next two hits are this, checking out the same page as the Irish visitor and the Amazon IP:

50.56.217.nn - - [22/Oct/2012:06:25:47 -0400] "GET /directory/example.html HTTP/1.1" 200 22861 "-" "Mozilla/5.0 (compatible; Embedly/0.2; +h**p://support.embed.ly/)"
50.56.217.nn - - [22/Oct/2012:06:25:48 -0400] "GET /favicon.ico HTTP/1.1" 200 19342 "-" "Mozilla/5.0 (compatible; Embedly/0.2; snap; +h**p://support.embed.ly/)"

This is Rackspace.

Then the last hit in this series of visitors (all looking at the same page) was this:

89.191.34.nnn - - [22/Oct/2012:06:30:01 -0400] "-" 408 - "-" "-"

You probably already have these guys blocked, but I thought the Irish visitor was interesting. Netname is COMPLETENETWORK and the RIPE desc is Routed Infrastructure Link Subnets. What do you make of that?

-- GG

[OnThePike]

# Amazon AWS/Elastic Cloud
deny from 8.18.144.0/23
deny from 23.20.0.0/14
deny from 46.51.215.0/25
deny from 46.51.215.128/26
deny from 46.51.215.192/27
deny from 46.51.215.224/28
deny from 46.51.215.240/29
deny from 46.51.215.248/30
deny from 46.51.215.252/31
deny from 46.51.215.254/32
deny from 46.51.128.0/18
deny from 46.51.192.0/20
deny from 46.51.208.0/22
deny from 46.51.212.0/23
deny from 46.51.214.0/24
deny from 46.51.216.0/21
deny from 46.51.224.0/20
deny from 46.137.0.0/16
deny from 50.16.0.0/14
deny from 50.112.0.0/16
deny from 54.224.0.0/11
deny from 63.92.12.0/22
deny from 63.238.12.0/22
deny from 63.238.16.0/23
deny from 64.15.138.160/27
deny from 64.15.156.64/27
deny from 66.7.64.0/19
deny from 67.202.0.0/18
deny from 67.205.69.32/27
deny from 70.38.0.0/17
deny from 72.21.192.0/19
deny from 72.29.185.0/24
deny from 72.44.32.0/19
deny from 72.55.128.0/18
deny from 75.101.128.0/17
deny from 79.125.0.0/16
deny from 87.231.235.2/32
deny from 107.20.0.0/14
deny from 174.129.0.0/16
deny from 184.72.0.0/15
deny from 204.74.108.0/24
deny from 204.236.128.0/17
deny from 204.246.160.0/22
deny from 204.246.167.0/24
deny from 204.246.168.0/23
deny from 204.246.176.0/21
deny from 204.246.184.0/22
deny from 207.171.160.0/19
deny from 208.47.248.0/23
deny from 209.201.96.0/22
deny from 216.137.32.0/20
deny from 216.137.48.0/21
deny from 216.182.224.0/20

[wilderness]

I've explained to you previously that your comprehension and/or use of these ranges is incorrect.

deny from 46.51.215.0/25
deny from 46.51.215.128/26
deny from 46.51.215.192/27
deny from 46.51.215.224/28
deny from 46.51.215.240/29
deny from 46.51.215.248/30
deny from 46.51.215.252/31
deny from 46.51.215.254/32
deny from 46.51.128.0/18
deny from 46.51.192.0/20
deny from 46.51.208.0/22
deny from 46.51.212.0/23
deny from 46.51.214.0/24
deny from 46.51.216.0/21
deny from 46.51.224.0/20

It would be more beneficial to your own use/comprehension to simply change the above lines to a single line (despite denying innocents), until you comprehend the proper use of CIDR ranges.

deny from 46.51.

FWIW, you've duplicated these comprehension errors in most everything you posted this morning.

[lucy24]

It would be more beneficial to your own use/comprehension to simply change the above lines to a single line (despite denying innocents)

I remember an earlier round of that discussion. I think users have to decide for themselves where their priorities lie. If it is important not to lock out

46.51.0.0/17
and
46.51.215.255
and
46.51.240.0/20

then that long block of lines is perfectly correct, no matter how unwieldy it looks. But I'd put them in numerical order with explanatory comments:

# allow first half of 46.51
deny from 46.51.128.0/18
deny from 46.51.192.0/20
deny from 46.51.208.0/22
deny from 46.51.212.0/23
deny from 46.51.214.0/24
deny from 46.51.215.0/25
deny from 46.51.215.128/26
deny from 46.51.215.192/27
deny from 46.51.215.224/28
deny from 46.51.215.240/29
deny from 46.51.215.248/30
deny from 46.51.215.252/31
deny from 46.51.215.254/32
# allow 46.51.215.255
deny from 46.51.216.0/21
deny from 46.51.224.0/20
# allow 46.51.240-end

But if the holes you're poking are that small, it might work better to use mod_rewrite or mod_setenvif. For example (don't cut & paste, this is off the top of my head)

RewriteCond %{REMOTE_HOST} ^46\.51\.(1(2[89]|[3-9]\d)|2\d\d)$
RewriteCond %{REMOTE_HOST} !^46\.51\.215\.255$
RewriteCond %{REMOTE_HOST} !^46\.51\.2([45]\d)

Note minor space-saving fudging because 2, >5, \d and 2, 5, >5 don't occur

[wilderness]

I remember an earlier round of that discussion. I think users have to decide for themselves where their priorities lie. If it is important not to lock out

Many thanks for your suggestion lucy, unfortunately if OnThePike doesn't adhere to stricter methods than some "putz beginner" is going to come along a day and/or a decade from now, and assume because nobody spoke up that these improper methods are proper.

FWIW, there's a couple of very long threads (A Close to Perfect Htaccess) that are filled with improper syntax errors (even for that period), because people copied and pasted mass sections into those threads that cannot today be edited out. (for the long range detriment of the thread it should be deleted completely, in spite of the valid references it contains.)

[lucy24]

there's a couple of very long threads (A Close to Perfect Htaccess) that are filled with improper syntax errors (even for that period)

Pah, that's nothing. I've found places on the actual apache* dot org site that contain the locution

(.*){exact-text-here}

I can hear g1 ranting now, can't you? :)

:: but seriously ::

That's why I wish there weren't such a strict time limit on editing posts. If I realize later that I gave bad advice or left out a crucial bit of punctuation-- or an equally crucial "not"-- the choice is between pestering a moderator and hoping nobody will notice. ("Between plague and cholera" as one recent post had it.)


* My fingers treacherously typed "amazon". I'm very, very glad I noticed in time.