Where are these coming from?

VERY bad spiders attacking my site...

[irock]

I tracked this IPs although I have no way to telling where they are coming from.

68.7.9.187 141,243 hits
66.27.117.110 52,330 hits

Pls tell me what they are... these spiders are consuming all my bandwidth.!

[jdMorgan]

irock,

68.7.9.187 Cox Communications, Inc, an ISP. Contact [e-mail]
66.27.117.110 Roadrunner, another ISP. Contact [e-mail]

You can look these up yourself using resources such as ARIN [arin.net].

Do you have any user-agents, or any other info available from your logs? Providing as much info as you can here will help to get you better answers.

Hope this helps,
Jim

[irock]

I will try to retrieve them... but what I don't understand is WHAT the hell they are trying to do? Download my site into their computer? Is it what they are trying to do? It's weird they have to access my site so many times. I don't understand their motives.

[jdMorgan]

irock,

Who knows? - Can't tell from what you've posted.

Can you see any pattern in the pages/files they loaded? Do you really have that many pages on your site, or are they retrieving the same pages repeatedly? Are all the pages they are retrieving valid (do they exist), or are these 'bots "shotgunning" your site, trying all possible variations of common filenames?

Again, the more info you provide, the better quality response you'll get here...

Whatever they are up to, they are loading pages at an excessive rate, and should be banned by IP for that alone, IMHO.

Jim

[SwissGuy]

Cox and Roadrunner are both famous as bandwidth providers for all sorts of masquerading robots which are everything except polite. I don't think that they are spin-offs from search engines but rather evil professional spam harvester or content merchandizing companies, usually based in the US (sadly, e-mail spam originating out of the US has reached a global level that is hard to accept - no other country comes even close by miles).

Such spiders are difficult to block based on their identity. One must use sophisticated and persistent monitoring software that tracks ongoing visitor sessions and possesses the ability to immediately block offending users based on a various rules, like a maximum request limit in your case.

In any other case and sadly enough, the only truly effective measurement is to reduce your seemingly excessive amount of documents available to the public ;)

[irock]

So, they are HARVESTERS? ARRRRGGGGGGHHHHHH

Why would they want my content? Are they trying to sell them w/o my consent?

And I don't think I have a lot of content fro them to grab.

[SwissGuy]

No idea, but maybe your web site is positioned so well for certain keywords that you caught those spiders' attention. Or maybe you enlist high-frequent keywords or conceptual themes that are in market demand (and will popup at a "e-learning content provider" :) ). Or your site is listed in a directory that is frequently visited by robots as a starting point (ie. business.com).

[irock]

...

What if I tell you they are all true in some degrees.

What is e-learning content provider? weird term indeed.

I JUST HOPE THEY ARE NOT TRYING TO HACK MY SITE OR ANYTHING...

[carfac]

Contact [e-mail]

Right. Like that will get you anything but an automated resonse and then silence. Don't waste your time, just lok rr out.

dave