Forum Moderators: coopster

Message Too Old, No Replies

PHP eating up memory

PHP memory

         

l3vi

11:40 pm on Jul 14, 2005 (gmt 0)

10+ Year Member



We had developed a porman's cron for a project. Very simple script that would check PS for itself then add itself if its not their.

The program has worked great, except one problem. Over time as it is doing it's while loop and sleep it starts eating up more and more memory until the server crashed. We just finished shutting down this script after it had grown from .01 in PS to 14MB over a couple days.

Any ideas on how to stop it from using more memory, or what is making it use more memory?


while(1<2) {
exec("pwd", $pwdval); // Find where we are at in the system
foreach($pwdval as $pwdstr) $pwdres.=$pwdstr;
$pwdreg=str_replace("/", "\/", $pwdres); // Format for preg_match.
exec("ps -A ux", $ps_return); // Get PS results
foreach($ps_return as $psstr) $psres.=$psstr;
if(preg_match("/".$pwdreg."\/thisscript.php/", $psres)) { } else {
system("/usr/local/bin/php ".$pwdres."/thisscript.php > /dev/null &");
// FAKE BROWSER TYPE
ini_set("user_agent","Lilith/1.0;");
$dh = fopen("".$OpenURL."","rb"); // Connect to remote system
while (!feof($dh)) {
fread($dh, 8192);
}
fclose($dh);
}
sleep($sleeper); // Sleep for x seconds.

[edited by: l3vi at 11:43 pm (utc) on July 14, 2005]

jatar_k

11:42 pm on Jul 14, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Welcome to WebmasterWorld l3vi,

have you tried cleaning up more

such as unsetting all vars at the end of each loop

l3vi

11:49 pm on Jul 14, 2005 (gmt 0)

10+ Year Member



Thank jatar_k,

If the script started over it would have null vars, while its in the loop I don’t think it would matter. Even on the $dh for fopen it would be overwritten when it passed thought the loop once more.

jatar_k

11:52 pm on Jul 14, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



strange though, I can't think of any other reason for the leftover

l3vi

12:03 am on Jul 15, 2005 (gmt 0)

10+ Year Member



Im going to apply unset to everything that I can, just in case. but this one gots me good.

Anyone know if there is some bug in the system command and long processes that gets linux to start giving more memory to an application if its running for a long time?

jatar_k

1:05 am on Jul 15, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



not sure

but, and this may be stupid, why not a real cron?

coopster

1:47 am on Jul 15, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



Check for includes(). Each iteration that pushes an include will push the code from the include into memory each and every time.

l3vi

3:28 am on Jul 15, 2005 (gmt 0)

10+ Year Member



coopster we have no includes. Im thinking it maybe the function or output that is building up data in memory. Im running test now to see if that is it.

jatar_k For our project it would require a large amount of work to use crontab as we would be adding 500+ entries a day as well as removing them at times. We needed something that was automated and would manage itself. This does the trick but is just building up memory over time.

I made some changes and checked PS after Jatar_k’s post. %MEM was 0.1 as of now it still is 0.1, don’t know how slow it is. One system that it is on builds up really fast in a couple days, on my system it took a week to get to 0.6 so testing is going to be slow.

Keep you all posted! ;)

l3vi

12:05 am on Jul 16, 2005 (gmt 0)

10+ Year Member



As of this point it has moved to .02 %MEM

Added code at command sleep()


sleep($sleeper);
// CLEAN UP VARS
unset($dh);
unset($pres);
unset($psstr);
unset($ps_return);
unset($pwdres);
unset($pwval);
unset($requestURL);
unset($pwdval2);
unset($pwdstr2);
unset($pwdres2);
unset($pwdreg2);
unset($CatName);
unset($array_total);
}

From what I see, after sleep() time the system starts the loop and builds up more memory.

So it must be something with fread, system.

As the out put is going to /dev/null system command would dump the output to /dev/null and that would be the end of that data. Unless /dev/null can build up a stack in memory till a program stops.

As we all know the data from fread can me set to a string var. Because Im not adding freads data to a var that would be my top choice if fread would store the data into memory as it cran the loop until the program is finished.

If the above does not fix it Im at the extent of my programming skills as my only guess would then be its something with how the system process php scripts that run indefinitely.

jatar_k

12:11 am on Jul 16, 2005 (gmt 0)

WebmasterWorld Administrator 10+ Year Member



I am glad testing is going to be slow, I have someone I want to ask about this and he is runnin around gettin married or some sillyness so he will be slow too. ;)

>> Unless /dev/null can build up a stack in memory till a program stops

I dont believe it can

I am going to have to replicate this and watch it

ergophobe

12:58 am on Jul 16, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I can't see when those concatenated variables ever get reset (like $pwdres and $psres). $psres still isn't getting reset to '' anywhere even with the unset() statements.

Don't these vars eventually ending holding a huge amount of data, maybe megabytes worth, or am I missing something?

l3vi

7:16 am on Jul 22, 2005 (gmt 0)

10+ Year Member



Hey jatar_k,

Has your friend gotten a chance to look at this?

We are still scratching our heads about it, and hoping someone can come up with a solution. ;)

alberthendriks

10:10 am on Jul 22, 2005 (gmt 0)

10+ Year Member



l3vi, Listen to ergophobe! He's pointing the problem exactly:
foreach($pwdval as $pwdstr) $pwdres.=$pwdstr;

Obiously $pwdres gets megabytes in length after a while if you don't reset it. Now you do reset it (unset), but you still have the following line which is now the cause of the memory overflow:

foreach($ps_return as $psstr) $psres.=$psstr;

I recommend the following line instead:

$psres = join("",$ps_return);

And instead of
foreach($pwdval as $pwdstr) $pwdres.=$pwdstr;
I recommend:
$pwdres = join("",$pwdval);

Then you don't need to unset anything (but still unsetting would do no harm).

I hope you see that this is the problem: .= adds something to the string, and at each while-iteration the string gets longer and longer.

l3vi

10:31 am on Jul 22, 2005 (gmt 0)

10+ Year Member



hehe.. I did not see his post. ;p

Will give that a try asap.

Thanks!

alberthendriks

10:44 am on Jul 22, 2005 (gmt 0)

10+ Year Member



Why don't you remove the "sleep"-line for testing? php will hang before you know (until you solve the bug using the solution above ;). btw I do agree with jatar that you should use a cronjob. Php scripts should have configured a maximum running time and a maximum memory usage.

l3vi

11:29 am on Jul 22, 2005 (gmt 0)

10+ Year Member



The time it would take us to setup each crontab we felt it would just take to much time for the project. We will be setting up about 10,000 of these a day ever day 7 days a week on many servers, and they will expire and be deleted at some point, so we did not want to deal with that either.

The simple solution was just to write a quick script that would active itself and run till it was gone as time and speed is what’s most important to us.

I will be removing sleep() while testing. ;)

alberthendriks

12:50 pm on Jul 22, 2005 (gmt 0)

10+ Year Member



er.. ehm.. about 70.000 a week? It sounds to me as something you should want to do right. In this solution, if php crashes for whatever reason, you'd have to restart all 70.000. Also you should be monitoring all 70.000 if they're still running individually.
What is the background of this? Is someone repeatedly deleting ps or something?

l3vi

3:29 pm on Jul 22, 2005 (gmt 0)

10+ Year Member



no no, not 70(dot), its 70 thousands. That’s what the script does it restarts itself, and monitor itself. We do not have to do anything to get it to restart. All we have to do is upload once, and it takes care of the rest till we delete it. ;)

l3vi

4:38 pm on Jul 23, 2005 (gmt 0)

10+ Year Member



Everything is looking good, in the fast-forward version with sleep of 2 I have found that it sits at 0.1 mem% until is finishes sleep and starts the loop over then it goes to 0.2. But after that it does not keep leaking memory even after a couple hundred loops.