4

I have a resource intensive script that takes about 30 minutes to complete. As its running it gradually consumes RAM. Once its finished it has consumed about 300mb. I need to run this script multiple times so this is becomming an issue. I have to restart the server every time i want to start running the script again as ram usage hits 100% and the script stalls.

How can i force php or mysql to release the memory? PHP Version 5.4.29 Centos VPS 2GB ram Max memory for a php scripts is set to 512mb.

The script makes a large amount of SQL queries. Should be in the 1000s. It also makes calls to 5 different APIs 50 times. It selects 50 row from a db table and loops through it, making calls to APIs etc to update it.

Garbage collection is enabled var_dump(gc_enabled()); returned true

 total used free shared buffers cached Mem: 2048 1522 525 0 0 182 -/+ buffers/cache: 1339 708 Swap: 128 0 128
Improve this question
8
  • 2
    are you running the php script via the shell? if so check what processes are left after your script ends. PHP is not a continually running process so should not be found in memory after the script ends. Commented Nov 29, 2014 at 20:27
  • i was initially running it via wget, but now im just running it through the browser. it isnt in memory after the script ends. mysql is showing up as using 4% ram and thats all i can see using memory. there doesnt seem to be any process using the ram. its just missing Commented Nov 29, 2014 at 20:45
  • Define "just missing". What makes you think the memory is not being released? (You may want to refer to linuxatemyram.com) Commented Nov 29, 2014 at 20:46
  • 1
    Could you show the results of free -m before running the script at all, and then after running it once, twice, three times, etc until we can see the pattern? Eventually when you run it the machine runs out of RAM? Commented Nov 30, 2014 at 14:47
  • 1
    Actually, to one-up that, how about the diff of top before and after the script runs? Please put special emphasis on the mysqld line! Commented Dec 1, 2014 at 4:30

4 Answers 4

Reset to default
2

When script is completed all resources should be freed, so you should not be in need of doing anything extra unless you use PHP/lib that seems to be buggy and leaks memory

EDIT

Memory leak is result of a bug and in languages like PHP if memory leak happens in PHP or its modules you will not be usually able to fix it other way by updating PHP or used modules to never version and hope this will come with leak fixed. You may try to narrow down what component is leaking and then report to its authors. As temporary solution you can work this issue around by restarting your httpd periodically which shall free all memory allocated by php or modules. But that's quick workaround only

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

How can i get to the bottom of a leak? or more to the point can i add code to my script that will allow me to clear memory that has leaked? i used the top command and there is nothing using this ram. Php is urrently using 200mb ram so its not php either.
0

You can set variable to null to free memory immediately.

I think you need to set up the database memory limit by adjusting the buffer size

How to set memory limit in my.cnf file

Improve this answer

Comments

0

This is not PHP which is taking up this RAM.

Once the php process is complete, it returns all of its RAM to the system. So this is not PHP which is causing this.

Some other application on the server is taking up additional memory, causing less to be available to php when it runs. From what you've described, it's easy to imagine that it's MySQL. You said you're running 1000's of queries. MySQL does profiling and caching internally to optimize those queries. This would absolutely cause it to take up additional memory after you've run the PHP script.

Also check on the other services. If any of the services are being called from your script, yet continue to run between script executions, they could also be taking a larger footprint in memory. Any of these services could be configured for smaller memory footprints, although perhaps some cannot.

Certainly, though, the solution is running PHP and MySQL on two different servers. It is far simpler to configure each application to use its entire server than to configure them to share one server nicely. 2 GB of RAM seems low for a script which requires 300mb to run.

If your server resources are fixed, it's possible you can modify the script to run longer but take less memory, too. Maybe you can load the largest data in pieces and flush each result out before loading the next. But to me, this would only be a temporary solution.

Improve this answer

Comments

-1

Check out the ob_implicit_flush() function.

PHP Docs

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.