Archive cron: ERROR (memory size exhausted)

Premium Features
1

hi all

I was looking for and reading various post about our porobl - here an elsewhere - before posting. nothing helped, though.

We ran our matomo <v4 for 3 years witouth major problems.

we use a cron, usually running every 2 hours (5 */2 * * * - we also tried different timings)

cron command:
php /var/www/vhosts/{servername}/httpdocs/console core:archive --url=https://{servername} > /dev/null
(I replaced our domain with {servername})
in matomo setting, we have set cron timing to 3600 seconds.

as said, this worked for <v4.

now we run:

Matomo-Version: 4.1.1
MySQL-Version: 10.3.27-MariaDB
PHP-Version: 7.4.15

after upgrading to v4, the errors began with archiving (I receive errors via email). First, there was errors with two plugins (UserFlow, MultiChannelConversionAttribution). We deactivated those and it ran without errors. We re-activated MultiChannelConversionAttribution without a problem. But after re-activating UserFlow, we got errors again. And now, even after deactivating the plugin again, errors dont go away.

short version of error message:

Fatal error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 274432 bytes)
(the 274432-part varies, a complete error message see below)

Now, 805306368 bytes is 768M, so we tried to increase memory_limit in php. We run a 32GB physical memeory server with just this one instance of matomo. and right now we have our php memory_limit configured at 8192M. I don’t know where this 768M in the error comes from. whatever we changed in php.ini, nothing changes in respect to the error.

this instance tracks several channels totalling at about 100k visits and 500k pageviews per day.

matomo frontend is not showing any errors (error messages from cron via email) and the data seems to update correctly.

any ideas? thank you!

example of error message (i get one every after every cron run):

Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=18&period=year&date=2021-01-01&format=json&trigger=archivephp.
Response was ' Fatal error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 274432 bytes) in /var/www/vhosts/{servername}/httpdocs/core/DataTable.php on line 1378 {"result":"error","message":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 274432 bytes) on \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/DataTable.php(1378)

#0 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(168): Piwik\\Plugins\\Actions\\Archiver-&gt;callAggregateMultipleReports() ","backtrace":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 274432 bytes) on \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/DataTable.php(1378)\n

#0 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(168): Piwik\\Plugins\\Actions\\Archiver->callAggregateMultipleReports()\n\n#0 [internal function]: Piwik\\Plugins\\CorePluginsAdmin\\Controller->safemode(Array)\n
#1 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(605): call_user_func_array(Array, Array)\n
#2 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(166): Piwik\\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n
#3 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(97): Piwik\\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n
#4 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(270): Piwik\\FrontController::generateSafeModeOutputFromError(Array)\n
#5 [internal function]: Piwik\\FrontController::triggerSafeModeWhenError()\n
#6 {main}"}{"result":"error","message":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 274432 bytes) on \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/DataTable.php(1378)

#0 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(168): Piwik\\Plugins\\Actions\\Archiver-&gt;callAggregateMultipleReports() ","backtrace":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 274432 bytes) on \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/DataTable.php(1378)\n

#0 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(168): Piwik\\Plugins\\Actions\\Archiver->callAggregateMultipleReports()\n\n

#0 [internal function]: Piwik\\Plugins\\CorePluginsAdmin\\Controller->safemode(Array)\n
#1 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(605): call_user_func_array(Array, Array)\n
#2 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(166): Piwik\\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n
#3 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(97): Piwik\\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n
#4 \/var\/www\/vhosts\/{servername}\/httpdocs\/core\/FrontController.php(270): Piwik\\FrontController::generateSafeModeOutputFromError(Array)\n
#5 [internal function]: Piwik\\FrontController::triggerSafeModeWhenError()\n#6 {main}"}'
2

Hi,

Can you make sure you edited the php.ini of your webserver and not the one of php-cli?

Also check if you restarted your PHP (e.g. php-fpm) afterwards.

3

I think it works correctly. We have to use Plesk to configure PHP, and I have to admit that it is not always entirely clear which php.ini is being edited.
but when I use a php script located in the same directory as matomo.php to print out memory_limit, it shows the correct value of 8192M

<?php
$memory_limit = ini_get('memory_limit');
if (preg_match('/^(\d+)(.)$/', $memory_limit, $matches)) {
    if ($matches[2] == 'M') {
        echo $matches[1].'M';
    } else if ($matches[2] == 'K') {
        echo ($matches[1]/1024).'M';
    }
}

output: 8192M

EDIT:
well, since we are on a shared server, I probably can’t change server-wide memory_limit. if I run php -i | grep "memory_limit in a console I get memory_limit => 256M => 256M

but this is also NOT 768M… so I don’t think we have a problem here.

oh, and about this:

I rebooted the server several times. and see above, my script can see the full amount of configured memory.

4

Ok I think i found problem and solution:

on our server, running core:archive as cron runs it in bash. and bash was configured (by the provider) using PHP 7.2 and memory_limit 256M.

After I figured this out, I could run the correct PHP version and with it the correct memory_limit. For now, it seems to work.

So it seems the two named plugins use some more memory but aren’t really a problem.

I still don’t know where the Allowed memory size of 805306368 bytes (768M) is coming from, though. It’s an odd value and I couldn’t find it anywhere in a .ini oder within matomo code.