Welcome to WebmasterWorld Guest from 23.22.220.37

Forum Moderators: Ocean10000 & incrediBILL & phranque

Message Too Old, No Replies

Log Files Filling up FAST.

Just switched servers. Filled up 5 gigs over the weekend.

     
1:23 pm on May 9, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 25, 2008
posts:88
votes: 0


So we run a pretty good webserver and we use rewrites throughout every piece of coding.

So now that we are on a new server the Log files are now filling fast. We did 5GB over the weekend. When they fill up the server does not run well. here is an example of what we are seeing.


65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomain.com/sid#1383a8c0][rid#13db4b40/initial] (4) RewriteCond: input='www.thisdomain.com' pattern='^thisdomain.com$' => not-matched
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomain.com/sid#1383a8c0][rid#13db4b40/initial] (4) RewriteCond: input='www.thisdomain.com' pattern='^www.thisdomain.com$' => matched
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomain.com/sid#1383a8c0][rid#13db4b40/initial] (2) [per-dir /home/user/public_html/] rewrite robots.txt -> http://www.thisdomainaswell.com/robots.txt
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomain.com/sid#1383a8c0][rid#13db4b40/initial] (2) [per-dir /home/user/public_html/] explicitly forcing redirect with http://www.thisdomainaswell.com/robots.txt
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomain.com/sid#1383a8c0][rid#13db4b40/initial] (1) [per-dir /home/user/public_html/] escaping http://www.thisdomainaswell.com/robots.txt for redirect
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomain.com/sid#1383a8c0][rid#13db4b40/initial] (1) [per-dir /home/user/public_html/] redirect to http://www.thisdomainaswell.com/robots.txt [REDIRECT/301]
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomainaswell.com/sid#137d2588][rid#13dbcc50/initial] (3) [per-dir /home/usernameb/public_html/] strip per-dir prefix: /home/usernameb/public_html/robots.txt -> robots.txt
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomainaswell.com/sid#137d2588][rid#13dbcc50/initial] (3) [per-dir /home/usernameb/public_html/] applying pattern '^(.*)$' to uri 'robots.txt'
65.11.111.12 - - [06/May/2011:10:22:06 --0500] [www.thisdomainaswell.com/sid#137d2588][rid#13dbcc50/initial] (4) RewriteCond: input='www.thisdomainaswell.com' pattern='^thisdomainaswell.com$' => not-matched


Should I just turn logs off?
1:58 pm on May 9, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2001
posts:5408
votes: 2


Should I just turn logs off?


Turning off your logs as the result of improper visitor activity is the wrong reaction.

What you need to do is determine the User Agents and IP ranges that are making these requests and take preventive action to deny them access.
7:16 am on May 10, 2011 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:10542
votes: 8


i would look for a RewriteLog directive in the server config file and either remove/comment it or set RewriteLogLevel to 0.
8:24 pm on May 11, 2011 (gmt 0)

Junior Member

5+ Year Member

joined:Aug 25, 2008
posts:88
votes: 0


Yes we disabled it. It wasnt from bad people. Our server just gets used.
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members