odd requests in access log

[curos]

So I run two different web servers off of apache and both have been getting these odd requests for [someone else's site] in their access logs. I'm not too familiar with access logs, but I thought it was weird that people could request stuff that is not on my webserver from my server. If someone could explain that to me, that'd be nice.

I guess what I am primarily concerned with is that one server reports success - 200. while the other reports 403 - forbidden. I think I want it to be 403, because they shouldn't be accessing other stuff anyway. Is there a setting somewhere that can be set so that both become 403? I'm thinking 403 is the preferred thing to have.

example requests below (ips *ed out):

***.***.89.148 - - [19/Aug/2005:19:10:18 -0700] "GET http://www.some_other_site.com/ HTTP/1.1" 200 41

***.*.205.116 - - [19/Aug/2005:19:58:37 -0700] "GET http://www.some_other_site.com/ HTTP/1.1" 403 3931 "-" "Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)"

[edited by: jdMorgan at 11:36 pm (utc) on Aug. 20, 2005]
[edit reason] Examplified, De-linked. [/edit]

[JAB Creations]

In Windows XP go to START and then RUN and type CMD and hit ENTER on your keyboard.

type...

ping some_other_site.com

Then do a ping on your own domain.

If you see the same IP address it may be that someone is hosting their site off the same server perhaps?

I've *NEVER* see a full length url request posted inside of an access log which is odd.

This ~MAY~ be access log spam. YES access logs get spammed.

Also you're gonna get docked for linking to that domain (even if unintentional).

I'm interested in seeing where this goes (about the log that is heh).

[edited by: jdMorgan at 11:37 pm (utc) on Aug. 20, 2005]
[edit reason] Examplified. [/edit]

[MattyUK]

If they were trying to use your server as a proxy via port 80 (rather than any other port) your access log would show their IP requesting the full URL of a thirdparty site.

Also a DNS error could produce this. If their systems DNS (or hosts file) thinks that XYZ.com domain should resolve to your IP then it might account for requests for other domains appearing in your access logs.

I have had the same problem and am working on some rewrite rules to knock all requests for a different domain on the head and direct them to a DNS_error.html page. The idea being that if I inform them that they have a DNS error or are trying to use me as a proxy then they shoudl stop or get it sorted out.

Would be interested to hear how your problem goes and if you find out anything else.

I should point out that in my access logs requests for other domains/files always return 404. The fact that you have a 200 would worry me.

[jdMorgan]

Quick post...

# BLOCK attempts to use our server as a proxy, but allow absolute URIs
RewriteCond %{THE_REQUEST} ^[A-Z]+\ /?http:// [NC]
RewriteCond %{THE_REQUEST} !^[A-Z]+\ /?http://([^.]+\.)?yourdomain\.com
RewriteRule .* - [F]

Jim

[MattyUK]

interesting I was going along the lines of:

RewriteCond %{HTTP_HOST} !^mydomain\.co\.uk [NC,OR]
RewriteCond %{HTTP_HOST} !^www\.mydomain\.co\.uk [NC,OR]
RewriteCond %{HTTP_HOST} !^000\.000\.000\.000 [NC]
RewriteRule .* [mydomain.co.uk...] [NC,L]

Obviously the 000 would be the servers IP.

I know the code has problems since it causes a infinite loop. But I haven't worked out what they are yet.

I need to allow relative urls as well hence going this route.

Any input gratefully received, I am much in need of sleep.

Cheers.

[jdMorgan]

The code I posted solves the problem described above. I've had it deployed for several years.

The problem is that the requested URI contains a canonical URL ("http://www.exampe.com/page"), so testing HTTP_HOST isn't effective. It is perfectly 'legal' under the HTTP protocol to include a full URL in a request header, but if it is not the URL of your site, and you see a 200-OK response in your logs, then your server is being used as a proxy.

I'm not sure what you mean by allowing relative URLs. All URLs are resolved by the browser to canonical URLs; It is the client (browser or robot) that "sees" relative URLs on a page, and decides what they mean.

Jim

[MattyUK]

Relative urls, my ignorance comes into play. I saw access log enteries for things like:

/<apage>.html
rather than:
[mydomain.co.uk...]

My (rather bad) assumption having given it only a few seconds thought was that relative urls did not need to be resolved.

Opps, of course they need to be if I think about it.

Thanks JD

[MattyUK]

Perhaps I can ask, would this be a good way to add https:// support?

RewriteCond %{THE_REQUEST} ^[A-Z]+\ /?http(?:s)?:// [NC]
RewriteCond %{THE_REQUEST}!^[A-Z]+\ /?http(?:s)?://([^.]+\.)?mydomain\.co.\uk
RewriteRule .* - [F]

Kinda new to mod_rewrite and regexp, so am grateful for any more detailed explanation.

Thanks

[jdMorgan]

The subpattern

/?https?://

will suffice, making both the leading slash and trailing 's' of '/https' optional.

Jim