Starting to worry....IP picking up pages automatically when I upload

169.207.238.197

[kstprod]

I'm getting a little worried here, as this IP seems to be automatically hitting brand new pages as I upload them.

169.207.238.197 - - [16/Nov/2002:08:33:07 -0500] "GET /newpage.html HTTP/1.0" 200 7982 "http://www.mysite.com/" "Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)"

All 4 times, they have hit the same "newpage.html" and all from the same IP. It resolves to a company called "Executive PC, Inc." Does anyone have any information on them? How can they be hitting that page seconds after I upload it?

Thanks,

Karen

[jdMorgan]

Karen,

By interpreting the contact email address in the ARIN listing as a URL, I reached an ISP in the MidWest. This sounds like someone has set up a "monitor changes" program to watch your site - maybe a competitor? Or possibly a site mirroring operation - Check the major search ingines to see if anyone is mirroring your content.

Put a block on that IP address and see what happens.

Jim

[bird]

Karen,
are you living in the Chicago area yourself? Do you look at your uploaded pages with your browser to verify that everything works? Your description makes it entirely possible that you're watching your own site checks... ;)

[andreasfriedrich]

That would be funny.

Andreas

[netcommr]

OrgName: Executive PC, Inc.
OrgID: EXPC

NetRange: 169.207.0.0 - 169.207.255.255
CIDR: 169.207.0.0/16
NetName: EPC
NetHandle: NET-169-207-0-0-1
Parent: NET-169-0-0-0-0
NetType: Direct Allocation
NameServer: E0.NS.VOYAGER.NET
NameServer: E1.NS.VOYAGER.NET
NameServer: E2.NS.VOYAGER.NET
Comment:
RegDate: 1995-05-26
Updated: 1999-12-14

TechHandle: MWM2-ARIN
TechName: Mittelstadt, Michael
TechPhone: [phone]
TechEmail: [e-mail]

Executive PC, Inc., New Berlin, WI 53151 US

[crash]

execpc appears to be an ISP :)

[kstprod]

Sorry for taking so long to respond. I am in the Chicago area, but I know for a fact it wasn't me, as I didn't even view the uploaded page online. It might be competitor who's ticked because I have been able to dominate most of the SERPS for most of our keywords, due to Fresh Bot. I'm not quite sure how to block the IP though. Maybe someone can explain. I have created an htaccess page for redirects, but thats all I know.

I do have another problem though guys. I think I'm making enemies or something. I have another IP, from the NC area, that is loading and reloading and reloading and reloading my pages at an unbelievable rate. I don't know whether they are trying to use a bunch of bandwidth, or are just trying to p*ss me off. The weird thing is this....this IP below ended up on my site in the first place by doing a search on Google for one of my main keywords. This happened on 11/14 and was a normal visit, they even signed up for my newsletter. THEN today, they came back and did all this weird stuff. From what I can tell by my logs, they came in today and manually typed in my url, like this www.MySite.com. Then 2 minutes after they arrived, there were 3 requests like this...

"GET /image.gif HTTP/1.0" 304 - "wysiwyg://37/http://www.mysite.com/page2.html"

then there was 7 minutes of this over and over...

152.31.32.65 - - [16/Nov/2002:16:09:45 -0500] "GET /page.html HTTP/1.0" 304 - "http://www.mysite.com/page.html" "Mozilla/4.08 [en] (Win98; U ;Nav)"
152.31.32.65 - - [16/Nov/2002:16:09:45 -0500] "GET /script.js HTTP/1.0" 304 - "-" "Mozilla/4.08 [en] (Win98; U ;Nav)"
152.31.32.65 - - [16/Nov/2002:16:09:45 -0500] "GET /image.gif HTTP/1.0" 304 - "http://www.mysite.com/page.html" "Mozilla/4.08 [en] (Win98; U ;Nav)"
152.31.32.65 - - [16/Nov/2002:16:09:46 -0500] "GET /page.html HTTP/1.0" 304 - "http://www.mysite.com/page.html" "Mozilla/4.08 [en] (Win98; U ;Nav)"
152.31.32.65 - - [16/Nov/2002:16:09:46 -0500] "GET /script.js HTTP/1.0" 304 - "-" "Mozilla/4.08 [en] (Win98; U ;Nav)"
152.31.32.65 - - [16/Nov/2002:16:09:46 -0500] "GET /image.gif HTTP/1.0" 304 - "http://www.mysite.com/page.html" "Mozilla/4.08 [en] (Win98; U ;Nav)"
152.31.32.65 - - [16/Nov/2002:16:09:46 -0500] "GET /page.html HTTP/1.0" 304 - "http://www.mysite.com/page.html" "Mozilla/4.08 [en] (Win98; U ;Nav)"

What are they trying to do?

Thanks for all the help guys,

Karen

<<added... I do have this persons email address, as they signed up for my newsletter, if it's real>>

[jdMorgan]

Karen,

wysiwyg://37/http://www.mysite.com/page2.html

Looks like this person opened your page using the Netscape Composer HTML editor. If they stayed around for several different page-views, it would indicate someone actually snooping, not just a "dumb" user who clicked the wrong selection while trying to bookmark your site or something (which does occasionally happen).

What server are you on? There are many threads here on blocking by IP address and user-agent, and you can block WYSIWYG access as well. How to do it depends on your server environment.

Jim

[kstprod]

jd,

I'm on Unix, and can modify my .htaccess. I've been reading on htaccess and using mod rewrite, but am lost as to exactly where to put the ban...I've seen everyone's ban list, but don't know enough about it all to add it. I'm too nervous I'd mess it up, so this is what I have now in my .htaccess

ErrorDocument 404 /notfound.html

So to ban an IP, my .htaccess would look like this, and only this? I'm not sure if this is right...

ErrorDocument 404 /notfound.html
deny from xxx.xxx.xxx.xxx
deny from xxx.xxx.xxx.xxx

Thanks,

Karen

[jdMorgan]

Karen,

Yes, that should work for specific single addresses. Here's a related thread [webmasterworld.com].

For more sophisticated bans, such as those which ban a block of addresses, I use mod_rewrite and extended regular expressions.

This would look like:

# Add options, Enable mod_rewrite
Options +FollowSymlinks
RewriteEngine On
# Block by IP address
RewriteCond %{REMOTE_ADDR} ^65\.97\.14\.251$ [OR]
RewriteCond %{REMOTE_ADDR} ^65\.102\.12\.2(2[4-9]¦3[01])$ [OR]
RewriteCond %{REMOTE_ADDR} ^194\.242\.43\.7[12378]$ [OR]
RewriteCond %{REMOTE_ADDR} ^195\.166\.(22[4-9]¦23[0-7])\.
RewriteRule .* - [F]

The above are just examples. The first one blocks a single IP address, the next three block ranges using three different notation methods:
65.97.14.251
65.102.12.224 - 65.102.12.231
194.242.43.71 - 194.242.43.78
195.166.224.0 - 195.166.237.255

(Note that in the last example, the entire possible range of final digits 0-255 is blocked because I don't specify that they have to match any particular value. In other words, I left the last digits out of the comparison, so they don't matter. Also, because this is the last RewriteCond in the list, there is no

[OR]

flag at the end of the line.)

Bottom line is that the deny from method will work fine for blocking specific addresses. When you get into ranges and very long lists, you will need stronger medicine. After that, you can get into auto-banning scripts and more fun stuff. :)

Jim

[jeremy goodrich]

I just got this in the log of one site:

169.207.238.197 - - [26/Nov/2002:12:34:27 -0600] "GET / HTTP/1.0" 200 10552 "http*//www.some-url/" "Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)"
169.207.238.197 - - [26/Nov/2002:12:34:27 -0600] "GET /page2.html HTTP/1.0" 200 13970 "http*//www.some-url.com/" "Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)"
169.207.238.197 - - [26/Nov/2002:12:34:28 -0600] "GET /page3.html HTTP/1.0" 200 15266 "http*//www.some-url.com/" "Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)"

And I'm pretty sure it's positiontech, because I just submitted those pages to them.

However, that guess is only because positiontech reverse looks up to an IP on the same block.

if it is them, they really, really need to identify themselves, because if it's not, I'd be very tempted to ban them.

And I don't want to ban a bot that resulted from me purchasing a service online...that would be bad.