Cron - php memory issue possible solution


(Pedro Estevão) #1

Hi,

My company start to use piwik for some time by now and everything is working as smooth as silk except the archive part.

We have try a cron with a wget to the url of archive.php and call the piwik console directly from a cron job. The both gave the sometimes the same output no matter what memory_limit configuration in php:

ERROR CoreConsole[2014-08-14 16:54:21] [c5b89] Got invalid response from API request: http://analytics.stg.online.xl.pt/index.php?module=API&method=API.get&idSite=5&period=month&date=last20&format=php&token_auth=3f5baeebed3ccb8951c93f2cb1c1e1a0&trigger=archivephp. Response was 'PHP Fatal error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 32 bytes) in /var/www/html/analytics/www/core/DataTable.php on line 1155 '

Have floowed some solutions found on this board and on some blogues but none of them seems to work nor the increse of memory_limit in php.ini configuration (by now we are using 2048M and the value on the error keep on show the 805306368 bytes).

Now we are running the cron passing the memory_limit value but it seems that in the recurrent requests made by the piwik console this variable is not passed to it. For exemple the first request is

/usr/bin/php -d memory_limit=2048M /var/www/html/analytics/www/console core:archive --url=http://analytics.stg.online.xl.pt/ > /var/www/html/analytics/logs/archive.log

but the next nested request goes as

/usr/bin/php -q /var/www/html/analytics/www/console climulti:request --piwik-domain=analytics.stg.online.xl.pt module=API&method=API.get&idSite=4&period=day&date=last2&format=php&token_auth=3f5baffbed3ecb8996c93f2cbbc1eca1&trigger=archivephp&pid=19fa5e574293c421b5208b0fb016d9e2909450cf8db8891010c289bcfc839902e8852c7ab9c691bb4f32dfc388143c33fa970.

Note that the memory_limit var is "lost".

It seems that is one recursive request that consumes all the memory. It would be a solution pass to it the memory_limit as in the first console call?

Just an ideia...

Pedro


(Matthieu Aubry) #2

please make the change of memory limit in the PHP config file if you can. Otherwise we could forward the parameters to the CLI, for this feel free to create issue on our issue tracker: Issues · matomo-org/piwik · GitHub


(Pedro Estevão) #3

Hi Matt thanks for your response.
I can change of memory limit in the PHP config file and I’ve made it but no matter what the value that I put in memory_limit, piwik keeps me saying the same memory value in the error message: 805306368 bytes. And this is the strange part of it no matter if I put 128M or 2048M…

Pedro


#4

what is the path of your php.ini?


(Pedro Estevão) #5

/etc/php53
why?


(Matthieu Aubry) #6

update the php.ini in /etc/php53/cli/php.ini ?


#7

/etc/php5/apache2/php.ini is for the PHP plugin used by Apache. This is the one you need to edit for changes to be applied for your Apache setup. do you have this on your particular setup?


(Pedro Estevão) #8

I have no /etc/php5/apache2/php.ini.
In my setup there’s only one php.ini availabla and is in /etc/php53. Verify that is the one in use by apache, using php_info.

My question is that no matter what memory_limit I put piwik archive always return the same memory value 805306368 bytes as I said before. Thus my suggested solution may not work because even I call archive trough console it seems to make a web request to /index.php?module=API&method=API.get&idSite=3&period=week&date=last8&format=php&token_auth=3f5caeexed3ccb8351c93f2cb1c1e1a0&trigger=archivephp wich make no difference beteween console and wget cron configuration. If you have memory problems with wget cron you will get it as well in console cron configuration.


#9

Would you by chance be using suhosin memmory setting?


(Pedro Estevão) #10

Think not. We haven’t it installed.


#11

what type of site is running? drupal? wordpress?

is there any sort of htaccess file that could be imposing a memmory limit?


(Matthieu Aubry) #12

did you restart the webserver after changing value ?

Also try to run this command:


php -i | grep "php.ini"


(Pedro Estevão) #13

lesjokolat, No we use our own CMS and entirely separated from piwik install thus different SO and configurations too.

matt, Yes I have restarted apache after php.ini changes.

The weird part is the value that appears in the message because it seems that php was using the 2048M configured but when the 2048M was exceded it keeps on showing the wrong memory value of 805306368 bytes in the error log output. And that was the reason of my post. Unfounded seems now :frowning:

Dispite that on the debugging tasks discovered that the console make some http requests. It was good that the console cron call don’t make any http request to proccess statistics since you can pass memory_limit to the command line that is going to be “lost” when the http request is made. And passing the memory_limit to console make possible to change this only when running the piwik cron tasks and keep a “logical value” in apache default configuration.

Pedro


#14

Do you have a htaccess file in the plugins folder? If so can you remove it and retry?

I also found an older post relating to date and timezone of php.ini

http://forum.piwik.org/read.php?2,113069,page=1#msg-113069

Could it be a regression of this error that is affecting yoursetup?


(Pedro Estevão) #15

lesjokolat think not. The php configuration already have a value for timezone.
We’ve simply update the php engine to the latest release 5.4.16 and now the error was gone with the exact configuration / php.ini used before…
Can’t tell for shure where the problem was :frowning: