Description
Description
This is an follow up on the issue hi-jacking I did earlier (sorry!). I haven't been able to replicate the issue in development or on our test servers during this time. So I tried a memory profiler on the production server, without any luck. Now I need some new advice from some experts!
#8646 (comment)
After running our application for about an hour we start to see an unhealthy amount of RAM usage. Fast forward a couple of hours and our sever will run out of RAM and start writing to disk instead.
PHP versions tested: 8.2.17, 8.2.14, 8.2.12, 8.2.9, 8.2.8, 8.2.7, 8.2.5, 8.2.4
Configuration: #8646 (comment)
sudo add-apt-repository ppa:ondrej/php
sudo apt update -y
sudo apt install php8.1-common php8.1-fpm php8.1-gd php8.1-mysql php8.1-mbstring php8.1-intl php8.1-xml php8.1-redis php8.1-curl
12 days with PHP 8.1.27 and we utilize 1.3 GB of RAM. Same utilization can be seen using PHP 5.6 - 8.1.
Around 3 hours with PHP 8.2.17 and we utilize 4.5 GB of RAM. This will build up to use all available memory.
I tried to use valgrind massif tool to locate the issue, but it's stable for 7 hours.
I have made two runs, using these settings, they give us the same behavior.
sudo valgrind --tool=massif /usr/sbin/php-fpm8.2
sudo valgrind --tool=massif /usr/sbin/php-fpm8.2 --nodaemonize --fpm-config /etc/php/8.2/fpm/php-fpm.conf
MB
4.480^ #
| # :::::::::::@:::::::::::::::::@:@:
| #::::: :: : ::: @:::: ::: :::: :::@:@::
| #: :: :: : ::: @:::: ::: :::: :::@:@::
| #: :: :: : ::: @:::: ::: :::: :::@:@::
| @#: :: :: : ::: @:::: ::: :::: :::@:@::
| @#: :: :: : ::: @:::: ::: :::: :::@:@::
| @#: :: :: : ::: @:::: ::: :::: :::@:@::
| @#: :: :: : ::: @:::: ::: :::: :::@:@::
| @@#: :: :: : ::: @:::: ::: :::: :::@:@::
| @@#: :: :: : ::: @:::: ::: :::: :::@:@::
| @@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| @@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| ::::@@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| :::::: @@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| ::::::::::::::: @@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| :@:: :::::: @@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| ::::@:: :::::: @@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| : ::@:: :::::: @@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
| ::: ::@:: :::::: @@@#: :: :: : ::: @:::: ::: :::: :::@:@:::
0 +----------------------------------------------------------------------->Mi
0 121.6
Number of snapshots: 52
Detailed snapshots: [6, 16, 17, 18, 19 (peak), 30, 45, 47]
@nielsdos You helped me out before and stated there was another way of using a memory profiler. I didn't see any memory leak using Valgrid thought. But these servers fail 100% of time not using Valgrid.
Did you observe memory leaking behaviour of this run? If yes, that means the problem is somehow not originating from a persistent allocation, and we can give instructions on how to include request memory inside the graphs.
- Is it still worth tracking request?
If not, I will start testing our application with PHP 8.2.0 to see if it's compatible before going the git-bisect route.
PHP Version
8.2.17
Operating System
Ubuntu 22.04.4