Somebody is messing with me?

Apace problem.

[krika]

Hi,
I'm new here.

I'va a website, where "requests currently being processed" is 200-300, every day.

But today it suddenly jumped to 700 and over. My site serves SWF files and i've hotlink protection, i think.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
RewriteEngine on
RewriteCond %{HTTP_REFERER}!^$
RewriteCond %{HTTP_REFERER}!^http(s)?://(www\.)?examplesite.com [NC]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

And i know that on the site there are only 200-300 people. Not even close to 700+

Maxclients is set to 768
Keepalivetimeout is set to 10 sec

I think people are playing on direct SWF file. But i don't know on which file.

And the problem is, how can i track down where from the requests are coming from? It's hurting my server and something needs to be done.

I'm newbie and i really need help :)
Thanks in advance.

[edited by: jatar_k at 5:25 pm (utc) on Mar. 30, 2006]
[edit reason] no urls thanks [/edit]

[StupidScript]

Welcome aboard, krika!

Check the server's log files. If there is one particular "person" doing all of the stuff you describe, then you should be able to see their activity pretty clearly.

You're looking for (a) any big group of requests from the same IP address and/or (b) lots of hits on any one file.

If you see any patterns, you should be able to get info about who (IP address, etc.) is doing the hammering. If you get this info, there are things you can do to block them.

[zomega42]

Another good place to look that not everyone knows about is apache's "server-status" page. You have to enable it in httpd.conf (it might also be possible in htaccess, I've never tried). It shows a list of all the active requests, what they're doing, and the ip of the visitor.

[krika]

Hi, thanks for the fast reply.

I've seen the logs, but i don't know how many is "much"

And the lines are different, like this:

[30/Mar/2006:10:41:10 -0500] "GET /thrumbs/filename1.gif HTTP/1.1" 304
[30/Mar/2006:10:41:10 -0500] "GET /thrumbs/filename2.gif HTTP/1.1" 304
[30/Mar/2006:10:41:10 -0500] "GET /thrumbs/filename3.gif HTTP/1.1" 304

etc.

Is apache showing "new line" for every JPG, GIF etc. like above.

[jdMorgan]

Each image, swf file, stylesheet, and external JavaScript is requested separately by the browser -- that's how the HTTP protocol [w3.org] works -- It is a 'stateless' protocol, and every request exists independently of the others.

A 304 status (described in the linked document above) indicates that the server responded with a Not Modified status to the client request. This means that the browser has the image in its cache, and is checking with the server. Since the server indicates that the file has not been modified, the browser will serve its cached copy, thus saving you the bandwidth of re-sending a file that has not changed.

Your problem is likely that the flash plug-in does not send an HTTP_REFERER heaqder -- most media players do not send this header. Therefore, your anti-hotlink code cannot block hotlinked requests for these files.

You can use a cookies-and-script approach to implement better hotlink protection that does not depend on the (very unreliable) HTTP_REFERER header. In this approach, pages on your site set a cookie, and the script checks for the cookie before serving swf and other media files. Thus, the cookie adds 'state' information to the requests that can be carried from request to request, above the stateless HTTP protocol.

An alternative is to use a server variable to define the path to your media files. Using PHP or SSI, you then include this variable path info in each link on your site. Then, once a day or once a week, you rename your media files directory, and change the server variable. So all links on your site still work, but hotlinks on other sites will break. Most Webmasters who hotlink don't like seeing broken links on their sites, and will be wary of hotlinking your files in the future. Of course, if the hotlinks are in forums or blogs that allow public posting, this mechanism doesn't come into play as strongly, but it still helps.

The approach above can be automated with a script that runs on demand or on a time schedule.

Jim

[krika]

Thanks Jim!
I'm smarter now. I think i choose the variable thingy.

I changed the folder name and the requests stopped. After changing the name back, requests increased again.

But does anybody know a site, where from i can learn it. It sounds easy, but PHP is not my profession.

[krika]

I used another way to replace the link.

update tablename set field = replace(field,'search_for_this','replace_with_this');

I updated the tabel fields and changed folder name.

Easy but effective.

Thank you again for helping.

[krika]

I just want to mention that the <hotlinker> is back. I had to change the /directory name/ again. And after changig a directory i had make a http restart becouse changing the directory seemed to not work instantly.

But is there ANY way to see who <snip> is hotlinking me. The hotlinker must have a big website. It is no dummy with his/her blog.

I mean programs for Linux, like "top" etc. Is there any to track the thief down?

Also i'm not sure anymore, if i'm being hotlinked or not.

[edited by: jdMorgan at 3:41 am (utc) on April 1, 2006]
[edit reason] Let's stay professionally technical, please. Thanks, [/edit]

[zomega42]

You need to see the "referer" in your logs. This is not always turned on in apache. If it was on, it would show up in your log files what url everyone was coming from. (Although as jdMorgan pointed out, if this is embedded media you won't see any referers -- if it is an actual swf file, that people click on and view, then you should see referers).

Try to get your host to tell you where your referer logs are, or have them turn on referer logging. Or, if that doesn't work, how about temporarily replacing the swf file with a php file that writes the referer to a file. Leave it that way for 10 minutes, then check the file to see what the referers were.

If you haven't looked up "server-status" yet you really should do that too.

[krika]

At the moment the problem is gone.

I have watched the logs, did not see anything, but maybe i watched wrong logs? I'm new to linux. I own the server. I can conf anything. I've not seen any referrars in my logs.

Maybe you can tell me how to turn it on, if it's off.

Thanks anyway, like i said, at the moment the problem is gone. And i can't replace the swf becouse i have several swf's and they are in different folders over the server.

I changed folder names and watched the apache 404 logs in live. I did'nt see a big increase for some file.(But i have several folders, i defenetly missed something).

[zomega42]

Here's the outline for turning on referer logs. (Really you should read the httpd.conf manual but these are the basic steps).

Open your httpd.conf
Find a line that starts with "LogFormat"
Add the following line near it:
LogFormat "%h %v %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

This defines a custom log type called "combined" that includes the referer field. Now either below that (if you are running a single website) or under your virtualhost directive (if you have multiple sites enabled), add a line

CustomLog /home/myuser/logs/access_log combined

[krika]

I viewed my httpd.conf and i found exact line.

But i'm runnig a DSM(zervex.com) and now i have to find the correct path where the logs with refferal URLs are.

Thanks for your help.
The rest is up to me i think :)