Hungry Bot

[menyak]

66.28.233.165 has had quite an appetite for my site recently. Does anyone know what it is?

[pendanticist]

I don't know 'what' it is, but I banned that puppy last week.

Along with:

66.28.23.147
66.28.68.237
66.28.139.25
66.28.233.165

...at various times within the last two weeks.

As for who it is, I can only say that:

RewriteCond %{HTTP_USER_AGENT} [e-mail] [NC,OR]
RewriteCond %{HTTP_USER_AGENT} [e-mail] [NC,OR]

At one time or another those IPs have masqueraded as either of the human-guided@ bots.

You'll notice:

12/15/02 20:23:19 IP block 66.28.23.147
Trying 66.28.23.147 at ARIN
Trying 66.28.23 at ARIN

OrgName: Cogent Communications
OrgID: COGC

NetRange: 66.28.0.0 - 66.28.255.255
CIDR: 66.28.0.0/16
NetName: COGENT-NB-0000
NetHandle: NET-66-28-0-0-1
Parent: NET-66-0-0-0-0
NetType: Direct Allocation
NameServer: AUTH1.DNS.COGENTCO.COM
NameServer: AUTH2.DNS.COGENTCO.COM
Comment: ADDRESSES WITHIN THIS BLOCK ARE NON-PORTABLE
Reassignment information for this block can be found at
rwhois.cogentco.com 4321
RegDate: 2000-10-12
Updated: 2001-12-05

TechHandle: ZC108-ARIN
TechName: Cogent Communications
TechPhone: [phone]
TechEmail: [e-mail]

OrgAbuseHandle: COGEN-ARIN
OrgAbuseName: Cogent Abuse
OrgAbusePhone: [phone]
OrgAbuseEmail: [e-mail]

OrgNOCHandle: ZC108-ARIN
OrgNOCName: Cogent Communications
OrgNOCPhone: [phone]
OrgNOCEmail: [e-mail]

OrgTechHandle: IPALL-ARIN
OrgTechName: IP Allocation
OrgTechPhone: [phone]
OrgTechEmail: "[e-mail]"@nospam.com

# ARIN Whois database, last updated 2002-12-14 20:00
# Enter? for additional hints on searching ARIN's Whois database.

...all those I've mentioned are within the same IP block.

I'll save you a bit of time by telling you that neither
'mapfeatures.net' or 'lerly.net' render any results in Google.

Just didn't settle well with me, so I banned 'em.

Pendanticist.

[wilderness]

I've had all of 66.28. Cognet denied for some time;

66.28.68.236 07/29/02 "larbin ([e-mail] <mailto:[e-mail]>)" Cogent Communications

even before this date I recall. Just don't have it documented.

[wilderness]

Anybody have any idea why Webmaster World converted the second parenthenses to a smiley?

[pendanticist]

Must have something to do with how this text editor interprets &gt and &quot; somehow.

That ain't all it does....

?

Within the text editor, the line above actually contains about 15 question marks.

Go figure...

Pendanticist.

[edited by: pendanticist at 2:04 am (utc) on Dec. 16, 2002]

[pendanticist]

Now I remember why I banned them. It's because of the ampersand. Something fishy about them being in the referrer string, or is it UA string. <shrug>

In the last few weeks I've seen four variants on that. The two mentioned above and two at yahoo.

Maybe someone who knows more about this can shed some light on how it is that those addies are set in there.

Pendanticist.

[wilderness]

Is it possible the amperand in the UA has some relation to a FTP bot and the password used by some ftp's?

I recall some bot in a mail reply a short time back giving me a line of wash about the possibility of ftp spidering.
Sorry I don't recall the details. JD and I discussed it and I sticky'd him the bots mail replies.

[pendanticist]

Is it possible the amperand in the UA has some relation to a FTP bot and the password used by some ftp'?

I don't know wilderness. But I suspect if menyak posted the string he questions, we might find it to be nearly identical to what I'm eluding to, in at least one regard. The ampersand in the string itself. Me thinks that's the key to this whole thing.

66.28.233.165 - - [13/Dec/2002:20:59:42 -0800] "GET /About_Awards_RWU_Cite.html HTTP/1.0" 403 224 "-" "Mozilla/4.0 [e-mail]"
66.28.233.165 - - [13/Dec/2002:21:06:10 -0800] "GET /Marketing_Yourself_R\xe9sum\xe9.html HTTP/1.0" 403 228 "-" "Mozilla/4.0 [e-mail]"

Notice that first GET? It represents the first 'hit' since I banned it. The second one intrigues me too, in that it asks for a file that has been obfuscated. That particular file has never resided within my domain.

That file should read /Marketing_Yourself_Resume.html.

This subject of 0bfuscating file names has come up in another posts and, I think it was JDMorgan, suggested that it might be related to the letters ( é ) in the word:

Résumé

While this word actually appears in my Meta and Title tags, it does not appear in the file name.

At any rate, I don't know how much, if anything it has to do the ampersand issue. But, they all come from the same IP block and to me that is very suggestive.

I've had several such requests as you can see.

66.28.68.237 - - [11/Dec/2002:13:51:10 -0800] "GET /robots.txt HTTP/1.0" 304 - "-" "Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6c ([e-mail])"

After I banned it:

66.28.139.25 - - [11/Dec/2002:16:27:39 -0800] "GET /Accounting_Forensic.html HTTP/1.1" 200 13699 "-" "-"

Notice the "-" "-"? I think they knew I'd banned the ampersand related part and proceeded to alter the UA so they could get thru.

Then, on another occassion I saw this:

66.28.23.147 - - [15/Dec/2002:06:32:58 -0800] "GET /Countries.html HTTP/1.0" 200 23364 "-" "Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6c [e-mail]"
66.28.23.147 - - [15/Dec/2002:06:35:45 -0800] "GET /Criminology_Corrections.html HTTP/1.0" 200 8025 "-" "Lynx/2.8.4rel.1 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6c [e-mail]"

Again with the ampersand.

You see, in my mind there is a correlation here and I want to understand exactly what it is. Especially if it's a nasty bot, which I suspect it to be.

(The chronology might be slightly in error, but the concept I've laid out is solid.)

Oh, before anyone suggests banning the entire IP block and forgetting the whole thing, let me say I don't want to...at this time. In the true spirit of Honey Pots, I'm gonna watch 'em and see just how creative they get whilst they look down on me as some kind of village idiot. :o

I recall some bot in a mail reply a short time back giving me a line of wash about the possibility of ftp spidering. Sorry I don't recall the details. JD and I discussed it and I sticky'd him the bots mail replies.

Please, let us know what the outcome is. I for one am interested.

Now, I turn it over to the night shift as it's past my beddy bye time.

<he said reaching for his blankie>

Let's put our collective heads together and get to the bottom of this one.

Pendanticist.

[menyak]

Hi guys - thanks for the replies. I think I'm still part of the night shift? Here's what my log file says...

66.28.233.165 - - [12/Dec/2002:12:14:10 -0500] "GET /robots.txt HTTP/1.0" 200 599 "-" "Mozilla/4.0 ([e-mail])"

66.28.233.165 - - [12/Dec/2002:12:15:10 -0500] "GET / HTTP/1.0" 200 28189 "-" "Mozilla/4.0 [e-mail]"

Pedanticist, what's the discussion about ampersands? Actually, I can't see one in your post!

[pendanticist]

Pedanticist, what's the discussion about ampersands? Actually, I can't see one in your post!

Uh, well, uh, would youu believe I was very, very tired and (here's where I must be Dyslexic) that I, um, <shuffling of feet> confused the '@' sign for an ampersand (&)?

'Cuz that's the absolute truth and I can't get in there to edit it! AaarGGhhhh! Do I feel foolish <blush>

---

I was right though. There's that yahoo addy. The '@' is the key to our mystery.

---

Boy, I sure would looooooooove to be able to get in there and edit that...

Hint, hint, hint

Pendanticist.

[menyak]

Believe it or not - I believe it! :) In fact, I even guessed it - but then the @ looked somewhat familiar in an email address... Anyway, there's a true '@' in MY logs.

[caine]

The company itself is selling high speed optical connections. No doubt providing every other web based service under the sun.

[pendanticist]

Anyway, there's a true '@' in MY logs.

Mine as well. You'll notice our log file entries are virtually one and the same?

It wasn't because of the ampersand that I systematically ban them, it's because of the @ sign.

Like I said earlier, that's the key to the whole thing.

---

(I hope this thread doesn't get too convoluted because of my stupidity. :o
<begin subliminal rhetorical message>
I don't suppose there's any way we can keep this just between you and me?
<end subliminal rhetorical message>

---

Pendanticist.

[pendanticist]

The company itself is selling high speed optical connections. No doubt providing every other web based service under the sun.

caine, you mean 'mapfeatures.net' and 'lerly.net'?

Like I mentioned, plug them into Google's Toolbar and you get absolutely nothing, nada.

Pendanticist.

[caine]

similar,

just put the company name into G, up it sprang with the bumf.

[pendanticist]

bumf

Not sure what that means.

Here's what I got:

[google.com...]

Anyway, I must be going. I'll check back in later on.

Pendanticist.