Archiving does not work - even if PHP limit is set to 1GB

robertharm · May 25, 2019, 3:56pm

We are running Matomo 3.9.1 and recently discovered that although auto-archiving via cron-job was configured, that this has not been triggered for several months.

Following your docs we tried to start archiving via php command line, unfortunately this even failed after assigning 1GB of RAM to php temporarily, with the following error(s):

INFO [2019-05-24 06:56:40] 20239 Error: Got invalid response from API request: ?module=API&method=API.get&idSite=14&period=year&date=last7&format=php&trigger=archivephp. Response was 'a:2:{s:6:"result";s:5:"error";s:7:"message";s:346:"Allowed memory size of 134217728 bytes exhausted (tried to allocate 1044480 bytes) on /piwik/core/DataTable.php(1344) #0 /piwik/core/ArchiveProcessor/PluginsArchiver.php(163): Piwik\Plugins\UserCountry\Archiver->callAggregateMultipleReports() ";}'

My hoster also informed me that the load during executing archiving was more than 30 (on a very powerful server). The sites tracked itself are not that huge, all sites together using this matomo instance track about 1000 unique visitors/day.

Are there any other memory settings (than memory_limit in php.ini) which need to be taken into consideration? Any other ideas on how to successfully trigger archiving would be much appreciated!

robertharm · May 25, 2019, 4:09pm

Issue has been solved - although our server had a PHP memory limit of 256M, when archiving was executed, the value from the constant minimum_memory_limit_when_archiving = "128M" was used - after changing this to 256M, archiving worked without any issues.

Lukas · May 27, 2019, 8:10am

Hi,

That shouldn’t really happen, the code behind this is really simple:

github.com

matomo-org/matomo/blob/14e8cf76bea0bfc004adfbc6817667fab3769691/core/SettingsServer.php#L143-L146


$minimumMemoryLimitWhenArchiving = Config::getInstance()->General['minimum_memory_limit_when_archiving'];
if ($memoryLimit < $minimumMemoryLimitWhenArchiving) {
    return self::setMemoryLimit($minimumMemoryLimitWhenArchiving);
}

Only thing I can imagine is that Matomo can’t properly detect the existing memoryLimit in your php.ini. Can you maybe check what this returns for you?

github.com

matomo-org/matomo/blob/14e8cf76bea0bfc004adfbc6817667fab3769691/core/SettingsServer.php#L184-L205


public static function getMemoryLimitValue()
{
    if (($memory = ini_get('memory_limit')) > 0) {
        // handle shorthand byte options (case-insensitive)
        $shorthandByteOption = substr($memory, -1);
        switch ($shorthandByteOption) {
            case 'G':
            case 'g':
                return substr($memory, 0, -1) * 1024;
            case 'M':
            case 'm':
                return substr($memory, 0, -1);
            case 'K':
            case 'k':
                return substr($memory, 0, -1) / 1024;
        }
        return $memory / 1048576;
    }


    // no memory limit

This file has been truncated. show original

robertharm · May 27, 2019, 5:22pm

mhhh, strange - var_dump(ini_get(‘memory_limit’)) is returning 256M on our server.

I have no idea why this did not work here…