homepage Welcome to WebmasterWorld Guest from 54.196.159.11
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Visit PubCon.com
Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
Forum Library, Charter, Moderators: Receptional & mademetop

Website Analytics - Tracking and Logging Forum

    
Is this a bot? Can I block it?
Suspected bot identifies itself as MSIE 7.0
avrofan




msg:3566743
 3:00 pm on Feb 5, 2008 (gmt 0)

I have a new website that gets an average of 3-4 real visits a day, plus numerous legitimate bot (Google, Yahoo, MSN etc.) vists.

Given the low traffic, one "visitor" stands out as unusual.

This visitor identifies itself as:

"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322)"

Each day it visits, starting with a "/" URI, then it fetches another dozen or so pages at the rate of 3-4 per second, clearly faster then a legitimate visitor.

The IP address is different for each visit. The last three visits used the following IP address's:

140.99.41.2
194.116.202.26
222.122.139.68

A check of the IP location etc reveals the following:

140.99.41.2 - UNITED STATES - DATABILITY SOFTWARE SYSTEMS INC - DERU.NET

194.116.202.26 - FRANCE - OPEN WEB SOLUTIONS - SERVERS.OWS.FR

222.122.139.68 - KOREA TELECOM - KORNET.NET

Has anybody else seen this? It looks like the IP adress and botname are false, so how can I block it?

Comments welcomed! If you need further information please ask.

Thanks :)

 

robsoles




msg:3567563
 11:42 am on Feb 6, 2008 (gmt 0)

Hey avrofan,

I've seen spambots using MSIE's footprint but not your situation.

At 3-4 pages per second it obviously is a bot, another way to tell if they are pulling pages at a slower pace is if they never fetch any of the graphics elements of your page, particularly not grabbing your stylesheet. Lynx browser and other text-only browers won't fetch any of that stuff either.

If I didn't want to just blacklist their IPs I would write a script to detect '3rd page inside a second' *or* 'msie fetch 3rd page without fetching support files' and effectively 'greylist' the IP address of the visitor for about an hour.

Regards,
robsoles.
Ps. Just blacklist their IPs.

avrofan




msg:3567668
 2:35 pm on Feb 6, 2008 (gmt 0)

Thanks for the reply, robsoles.

Another IP address used today:

72.52.207.94 - UNITED STATES - LIQUID WEB INC - LIQUIDWEB.COM

I hesitate to use blocking by IP, as it seems they have a range of IP address's to use. Each day another one appears!

I liked your suggestion of time based blocking - I'll look at writing a brief PHP script for it.

I'm still curious about who they are, and what they are doing. I'd be interested to know if anyone else has had a similar experience.

Regards

avrofan

Krispy2




msg:3577899
 2:43 pm on Feb 18, 2008 (gmt 0)

>> Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 2.0.50727; .NET CLR 1.1.4322)

I've been seeing this from something that is trying to inject its own URL's into the "&" parameters in my URLs.

See also: [webmasterworld.com...]

Bewenched




msg:3590561
 4:09 am on Mar 4, 2008 (gmt 0)

These same guys are attempting to do link injection

69.16.252.140 is what they were using today
OrgName: Liquid Web, Inc.

johnh123




msg:3602137
 4:15 pm on Mar 16, 2008 (gmt 0)

What is this about? Liquid web is, or at least appears to be, a legitimate web host. I was thinking about using them, actually, when I ran a search and find this.

jdMorgan




msg:3602150
 4:45 pm on Mar 16, 2008 (gmt 0)

The Web hosts you should not block are those used by your linking partners. Consider it this way: Why would another server be fetching pages from your site?

You expect referrals from other servers (and almost always by domain name, not IP address), but you don't expect page requests from other servers unless a site that links to yours is running a script to check the validity of their out-going links.

Other than that, you can safely block any IP address range that resolves back to "servers" or "hosting."

For a requests-per-second-based blocking script in PHP, see Blocking Badly Behaved Bots [webmasterworld.com] (third of three parts).

Jim

Badger37




msg:3603952
 1:02 pm on Mar 18, 2008 (gmt 0)

I think this is similar to the problem I've been trying to deal with in this thread: [webmasterworld.com...]

Searching on Google for some of the addresses mentioned earlier i.e. 69.16.252.140 shows a post titled 'someone's scraping me' this page lists numerous addresses that seem to have been taken over by some malicious s/w that tries to inject URL's in to peoples logs/code.

It's possibly a game or an application that the same people have downloaded which infects their servers.

It seems quite a lot of people are seeing this but I haven't found an answer yet.

Blocking the IP addresses doesn't seem to be the answer as more and more PCs will become infected and the list would grow unmanageable.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / WebmasterWorld / Website Analytics - Tracking and Logging
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved