Dragon135

Suspicious crawler leading back to strange open directory

[Maintainer]

Hi,

My site is being hit by an unknown agent from 69.64.46.84. The ip address resolves to [dragon135.startdedicated.com...]

It's an open index that contains a large gunzip file, and a directory, "rue8", which has a bunch of php scripts and a "google" directory containing a large number of huge text files full of urls.

Does anyone know what's going on here? Searches for the ip address indicate it's a google bot, but I doubt google would be so sloppy.

Is this a nefarious site or just someone experimenting with spiders?

[Mokita]

Hi,

Welcome to Webmasterworld.

The IP you have provided actually resolves to SERVER4YOU /
Hosting Solutions International, not Google. What tool did you use to get Google?

[Maintainer]

No tool. I did a web search and saw a lot of references to stuff like this:
Googlebot/2.1 (+http://www.googlebot.com/bot.html)|69.64.46.84
I'm not technical minded and don't know of any other methods.

[Mokita]

Okay - you need to use reputable online tools to do a Whois Lookup or get IP Information.

I mostly use DNSstuff.com, but occasionally use arin.net or other Internet Registries. There are lots of others around though, so you can take your pick.

Is this a nefarious site or just someone experimenting with spiders?

There is no way of really knowing unless you can contact someone involved with the bot (which is highly unlikely).

Personally, as it is crawling from a hosting server, with no identifying information, I'd be blocking it in .htaccess pronto and not even think twice about it.

But as always with bot blocking - you have to decide based on your own circumstances. There is no "one size fits all" block list.

[Maintainer]

The first thing I did was block it with a rewrite rule in .htaccess. What's freaking me out is the hundreds of megs of urls in those directories and the php scripts which I can't figure out.

[Maintainer]

I'm searching through the url list on that site and the very same pages that I've been getting hit by comment and referrer spam on are the ones listed there. I may have found the root of my site's scraper and spam issues.

Errrrg.

[dstiles]

Block all "servers". In this case block the whole of the Codero range 69.64.32.0 - 69.64.95.255

There are lots of server ranges that should be blocked this way.

[Maintainer]

@dstiles

Thanks, I'll do that.