strange referral string from atgi.net

file:///C:/MetaTags/add_0005/add_0003.chp

[idiotgirl]

This is showing up in log files for multiple domains:

66.182.20.91 - - [02/Aug/2002:13:42:12 -0400] "GET / HTTP/1.1" 403 2462 "file:///C:/MetaTags/add_0005/add_0003.chp" "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)"

Just one single hit per domain. I don't know if they're adding to their index or what. This traces to atgi.net, which refreshes to callatg.com. It looks like a dialup service, but it's hard to tell exactly what they're up to. My .htaccess served a 403 so they basically got nothing - but I'm curious if anyone else has seen this.

[NeedScripts]

Under general circumstances it is useless to block dialup IP and generally it will do you more harm than good.

1) Many times people using dialup will have different IP every/some/few time they go online.

2) If I am not wrong, they can use more than 1 dial up number and hence the IP will be changed.

3) By blocking dialup IP, you might be actually blocking other good clients/customers/surfers, Cuz, Dialup companies uses 1 IP to serve tons of customers.

what is file:///C:/MetaTags/add_0005/add_0003.chp

I believe someone is using a program that will download Meta Tags [or other info] from the web sites in the list/search results for any particual phrase. And the referral to your web site, might from the output report of the program.

Again, I might be wrong ;)

Hope this will help.

[Key_Master]

.chp is a Chapter File extension for Corel Ventura [corel.com]. The file://.* is the referring URL from the visitor's computer. I'm suspicious of such referrers.

By blocking dialup IP...

"idiotgirl" is blocking the referrer- not the IP. She's much smarter than her profile name suggests (not to suggest that blocking dialup IPs is a bad idea). ;)

[jdMorgan]

K_M,

Yeah, "file:" and "wysiwyg:" are both unwelcome referers here...

Your last line gets a big "LOL"... Crazy over image stealers, maybe. Idiot? No.

Idiotgirl,

I haven't seen that one yet, but glad I'll 403 'em on the referer, too.

Jim

[bird]

"file:" and "wysiwyg:" are both unwelcome referers here...

So if I put a link to your site into a file on my local disk, and access it from there, I'm not welcome to you anymore?

A "file://" referrer is pretty much aequivalent to a bookmark. Do you block users accessing your sites from their bookmarks as well?

[jdMorgan]

bird,

Browsers are welcome.
Bookmarks are welcome.
Legitimate SE spiders are welcome.
Research spiders from edu domains are welcome.

Proxy caches are welcome if they pass a referer. (I try to contact those who don't)

Others, maybe. Depends on the remote_host, remote_IP, page requested, and referer, too.

Site-suckers and Spambots need not apply.

We have a tight budget. As long as we have to pay the bandwidth bill, we make the rules. I wish it
didn't have to be this way - it would save me several hours a week. But we've been abused and had
to pay a BIG penalty. Not again...

Jim

[bird]

"file:" and "wysiwyg:" are both unwelcome referers here...

Browsers are welcome.

I have yet to see a non-browser pass on a "file://" referer. For a spider developer, that would require some completely useless extra coding, at the risk of sticking out in your logs. Not very likely to happen.

[Key_Master]

Most of the file://.* referrers I see come from directories containing some form of /email scavenger/ in the filename.

It is concievable that the visitor is drafting up Web pages on their own computer and has included a link to your own site. However, I hardly consider feeding this referrer a 403 as rude.

I'm curious bird, do you consider blocking image links in poor taste?

[bird]

Most of the file://.* referrers I see come from directories containing some form of /email scavenger/ in the filename.

Ah well, but in this case, the "file://" part is really irrelevant, isn't it?

I often see people promote blocking everything they can't explain (I'm not saying you do this). In another current thread, someone proudly presents a .htaccess configuration that blocks iCab (probably the best browser for the Mac), and Proximotron (Bretts choice of personal proxy server), and a number of other useful and legitimate tools.

My own philosophy is rather to allow as much as possible, and only to block those things that provably cause damage, in terms of bandwidth, address harvesting, etc. I never block some UA or IP after only a handful of accesses, just because they "look suspicious". To me, that's simply a matter of common sense and effective use of my time and resources.

I'm curious bird, do you consider blocking image links in poor taste?

I'm not sure what you mean here. If you're talking about image leeching (theft of both copyright and bandwidth), I have installed redirects in such cases before.

[Key_Master]

>>>Ah well, but in this case, the "file://" part is really irrelevant, isn't it?

No, not really. To see this referrer means a direct link to your site has been made from deep within the visitors computer. This usually indicates a visit from someone with a fair amount of technical experience and or that portions of your site has been downloaded to the visitors computer.

Keep in mind, this only blocks that specific request. The IP hasn't been blocked and the visitor is still able to traverse through your site through normal site links. A good 403 error response will include a link to the home page.

<added>

In another current thread, someone proudly presents a .htaccess configuration that blocks iCab (probably the best browser for the Mac), and Proximotron (Bretts choice of personal proxy server), and a number of other useful and legitimate tools.

I agree! Some people take it too far. ;)

[idiotgirl]

My reason for concern was the fact that this same referral string showed up in multiple domains, a few of which are not even "live". Because of the meta tags filename on someone's local system, to me it has the earmarks of a bull-in-a-china-shop type data grabber. (robots.txt was never requested for this UA.)

I don't like those.

And until I required that images only be served to people actually visiting the domain(s), I had several that had a total of thousands of offsite image calls a week, no to mention people who were compelled to download entire sites with extensive image galleries for their offsite viewing pleasure. Some of these images were later found in online galleries, as well as traded all over the internet for other purposes. Enter: the copyright dilemma. (No thanks.)

I've never felt theft, intentional or otherwise, was a form of flattery. For those who wish to argue about how harmless it is, and pontificate about what a free and wonderous place the internet is, or chastise me because I should "expect" to be stolen from... they can tell their story walking. They can choke on a 403 until Hell freezes over. They are not a unique and special snowflake. In a bar? I'd knock them off their chair.

Therefore, the

file://

was in integrated in my .htaccess. My 403 serves a link for real live visitors to enter the site through its front door, with the ability to email the admin (me) from that page if there is a problem. Harvesting programs most likely don't even see that.

There are several cases where my nic is entirely appropriate. Like when I'm choking with a chunk of Perl at 3am. I admit I'm not the sharpest knife in the drawer for such things, but I do surprise myself sometimes. I am constantly trying to learn and refine what I've learned through newsgroups, WebmasterWorld, and the occasional friendly hacker. I'll take it anywhere I can get it. Perhaps I have an unhealthy obsession with things like htaccess and bots, but it's an ongoing process of evolution I enjoy - as is the refinement of the elements.

I need to get out of the house more

[jdMorgan]

Ditto post 11, paragraphs 4 & 5 in their entirety. And colorfully stated, too!

Jim