Spider or ripper?

[tangor]

Yesterday this came through and took 5,000 pages

The IP was 76.203.212.nnn and whois shows:

AT&T Internet Services SBCIS-SBIS-6BLK (NET-76-192-0-0-1)
76.192.0.0 - 76.255.255.255
RCSNTX ADSL BRAS29 PPPoX SBC-76-203-208-0-21-0703192346 (NET-76-203-208-0-1)
76.203.208.0 - 76.203.215.255

What makes it interesting is the rotating UAs used during the session. Number first is how many times it was used/changed.

836 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
816 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
862 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)
859 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; InfoPath.1)
810 Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET)
809 Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7

I don't have any speed traps set up, so this one got by. First one I've seen like this which was not one of the site rippers out there.

[wilderness]

These PPPoX's of ATT-SBC have long been a problem.

At one time, I had most every PPPoX I could find ranges on, denied.

I get an unusual amount of traffic from ATT-SBC IP ranges which through ARIN are shown as a Texas locale, however when I do a tracert or a google on the org., name, it ends up somewhere else.

Many thanks for the heads up.

[jdMorgan]

Since the SBC Network Operations Center is here, the fact that all or almost all SBC traffic apparently comes from Richardson Tx is no more meaningful than the fact that all or almost all AOL traffic seems to originate from Sterling Va.

Joke: You could probably reject those MSIE requests, based on the fact that they are missing the usual two-dozen ".NET CLR" tokens... :)

Jim

[wilderness]

Since the SBC Network Operations Center is here, the fact that all or almost all SBC traffic apparently comes from Richardson Tx is no more meaningful than the fact that all or almost all AOL traffic seems to originate from Sterling Va.

I think all those ec2 folks are in Texas as well ;)

[dstiles]

jdMorgan NET CLR - unfortunately not unless there is secondary evidence in the headers. I have a client some of whose customers come in on basic MSIE UAs.

In fact my own (up-to-date but almost never used) MSIE 6 on Win2000 Server reports the UA Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0). No idea why but it's a problem checking for the baddies amongst the goodies who come in thus attired.

Whilst I get quite a few AT&T "attackers I get far more from comcast.