512M in PHP memory_limit configuration settings, is it too little?

Support & Bugs
1

I received this on my email from the matomo cron:
ERROR [2020-12-29 17:05:56] 1266411 Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=2&period=day&date=2020-12-29&format=json&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log

The memory_limit in my php.ini is at 512M, isn’t it already too much? Isn’t that enough for matomo?

2

Hi,

To little memory is just one of many reasons why this could go wrong.
The only way to know why is to check the error log of your server for the exact error that occured.

3

Not sure how relevant this will be but just a couple of steps I’ve tried to resolve the same issue myself.

Scenario: Cloudlinux on Cpanel server with CentOS 6.10

  1. Updated available RAM in php.ini manually to 1024Mb via
    /var/cagefs/[foo]/[foo]/etc/cl.php.d/alt-php[ver]/alt_php.ini

  2. Changed the Cron line from:
    php /home/[foo]/public_html/console core:archive …
    …to…
    /usr/local/bin/php /home/statsoct/public_html/console core:archive …

… to ensure the correct PHP version was being run as per:

The memory increase on its own doesn’t appear to have made much of a difference so waiting to see if specifying the PHP engine path directly makes it work.

Side note: Also checked mod_security for issues and can’t find anything that might be suspect there…

4

As an aside, I noticed that the error log was bringing up one particular site with problems.

I visited the site in question in the Masomo system to check if it was erroring out there… Nope… visited the actual site for any issues in the wordpress plugin, etc… Nope…

… and yet, since then the error has since disappeared.

Part of me wonders if there’s a data glitch somewhere that needs a nudge before it’ll clear out and allow the other “fixes” (read: desperate attempts to fix) to work.

5

… and one last observation.

The error pages were being left open with active connections on the server, so for every error we got we’ve find there was load being generated by the /[matomo]/index.php that just sat there. I’m guessing the 500 response generated an instance of apache but that’s just a guess.

Once I’d applied the fixes/steps above and killed the remaining connections to the index.php, the orphaned processes/connections didn’t recreate themselves and load has stayed significantly lower since.

6

I changed memory_limit to 1G in php.ini and run manually the cron, and it’s seems to work witout errors.

These are matomo error logs:

sudo grep "matomo" /var/log/php-fpm/www-error.log
[27-Dec-2020 15:17:27 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '608907',   1 => 26,   2 => 'x%����',   3 => '68546',   4 => '68546',   5 => '253048',   6 => '2020-12-27 15:17:27',   7 => 'W1Vt5H',   8 => 2,   9 => 155,   10 => 8187,   11 => 2405,   12 => 1226,   13 => 1609082146140,   14 => '253048', )
[28-Dec-2020 00:36:38 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_network' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '616689',   1 => 4,   2 => '��7�$y��',   3 => '226483',   4 => '51812',   5 => '51811',   6 => '2020-12-28 00:36:38',   7 => '0cxH0P',   8 => 2,   9 => 19,   10 => 1496943356,   11 => 1785,   12 => 182,   13 => '226482', )
[28-Dec-2020 07:06:04 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '620729',   1 => 23,   2 => '&� ��V�	',   3 => '122626',   4 => '122626',   5 => '122625',   6 => '2020-12-28 07:06:04',   7 => 'rMNwMo',   8 => 2,   9 => 1023,   10 => 2765,   11 => 2180,   12 => 283,   13 => 1608938301204,   14 => '122625', )
[28-Dec-2020 07:09:09 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '620729',   1 => 23,   2 => '&� ��V�	',   3 => '122626',   4 => '122626',   5 => '122625',   6 => '2020-12-28 07:09:09',   7 => 'qDEDCT',   8 => 3,   9 => 185,   10 => 874,   11 => 2180,   12 => 283,   13 => 1608938301263,   14 => '122625', )
[28-Dec-2020 08:14:57 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '621636',   1 => 24,   2 => 'P� ��8�',   3 => '36162',   4 => 0,   5 => 0,   6 => '2020-12-28 08:14:56',   7 => 'snGG7V',   8 => 1,   9 => 0,   10 => 2351,   11 => 2378,   12 => 413,   13 => 1608938300822,   14 => '36161', )
[28-Dec-2020 13:29:43 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_dom_processing' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '625031',   1 => 24,   2 => '�D��3#�',   3 => '7411',   4 => 0,   5 => 0,   6 => '2020-12-28 13:29:43',   7 => 'G1wWeG',   8 => 1,   9 => 0,   10 => 18219268,   11 => 0,   12 => 0,   13 => 48,   14 => '7410', )
[28-Dec-2020 15:35:46 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '626721',   1 => 25,   2 => ':_V�',   3 => '2613',   4 => '2613',   5 => '229934',   6 => '2020-12-28 15:35:46',   7 => 'vTL2dl',   8 => 2,   9 => 28,   10 => 447,   11 => 818,   12 => 213,   13 => 1608216851288,   14 => '229934', )
[28-Dec-2020 16:35:38 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_network' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '627604',   1 => 26,   2 => 'e�������',   3 => '417631',   4 => '30278',   5 => '235758',   6 => '2020-12-28 16:35:38',   7 => 'afzDnf',   8 => 2,   9 => 19,   10 => 3210,   11 => 1439290969,   12 => 3262,   13 => 156,   14 => '417630', )
[28-Dec-2020 18:19:48 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_network' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_network, time_server, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '629476',   1 => 4,   2 => '��2��',   3 => '12863',   4 => '24196',   5 => '24195',   6 => '2020-12-28 18:19:48',   7 => 'VhrdJh',   8 => 2,   9 => 19,   10 => 1433149099,   11 => 634,   12 => '12862', )
[29-Dec-2020 00:01:39 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '633836',   1 => 37,   2 => '��2�=�',   3 => '101203',   4 => '101203',   5 => '101202',   6 => '2020-12-29 00:01:39',   7 => 'V9OzaK',   8 => 2,   9 => 590,   10 => 1138,   11 => 608,   12 => 434,   13 => 1609145263746,   14 => '101202', )
[29-Dec-2020 01:30:46 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '634584',   1 => 21,   2 => '�S�C' . "\0" . 'J�',   3 => '7517',   4 => '76317',   5 => '76316',   6 => '2020-12-29 01:30:46',   7 => 'UrL0ND',   8 => 4,   9 => 515,   10 => 1034,   11 => 910,   12 => 1733,   13 => 1609195609572,   14 => '7516', )
[29-Dec-2020 16:13:26 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?' . "\0" . '��y��a',   3 => '139042',   4 => '139042',   5 => '139041',   6 => '2020-12-29 16:13:26',   7 => 'okWmSl',   8 => 3,   9 => 97,   10 => 169,   11 => 733,   12 => 244,   13 => 1608863484396,   14 => '139041', )
[29-Dec-2020 16:45:04 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?' . "\0" . '��y��a',   3 => '468152',   4 => '468152',   5 => '468151',   6 => '2020-12-29 16:45:04',   7 => 'o8euq1',   8 => 9,   9 => 1568,   10 => 198,   11 => 426,   12 => 204,   13 => 1608863484729,   14 => '468151', )
[29-Dec-2020 18:00:28 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?' . "\0" . '��y��a',   3 => '455643',   4 => 0,   5 => 0,   6 => '2020-12-29 18:00:28',   7 => '1TnS7X',   8 => 1,   9 => 0,   10 => 265,   11 => 5345,   12 => 226,   13 => 1608863492133,   14 => '455642', )
[29-Dec-2020 18:23:23 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?' . "\0" . '��y��a',   3 => '181951',   4 => '181951',   5 => '181950',   6 => '2020-12-29 18:23:23',   7 => 'AySWXB',   8 => 6,   9 => 663,   10 => 354,   11 => 422,   12 => 564,   13 => 1608863508082,   14 => '181950', )
[29-Dec-2020 22:33:26 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '649569',   1 => 26,   2 => '�
                                                                     �%�*r',   3 => '63778',   4 => '63778',   5 => '263630',   6 => '2020-12-29 22:33:26',   7 => 'D0HuYX',   8 => 4,   9 => 590,   10 => 1562,   11 => 380,   12 => 471,   13 => 1607699349150,   14 => '263630', )
[30-Dec-2020 15:38:30 UTC] Error in Matomo (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'time_transfer' at row 1 In query: INSERT INTO matomo2_log_link_visit_action (idvisit, idsite, idvisitor, idaction_url, idaction_url_ref, idaction_name_ref, server_time, idpageview, pageview_position, time_spent_ref_action, time_dom_processing, time_network, time_server, time_transfer, idaction_name) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) Parameters: array (   0 => '661102',   1 => 3,   2 => '�������',   3 => '477726',   4 => '477726',   5 => '477725',   6 => '2020-12-30 15:38:30',   7 => 'C7e74L',   8 => 2,   9 => 106,   10 => 354,   11 => 699,   12 => 179,   13 => 1609226989270,   14 => '477725', )

Did you see something?
(Weird characters are probably emoji)

P.S.
I found this in the control of the matomo system:

UTF8mb4 charset

Your database supports the utf8mb4 charset, but your database tables have not yet been converted. You can do this by running the /var/www/html/matomo2/console core:convert-to-utf8mb4 command or by turning on automatic conversion in the General Settings.

This is required to retain 4-byte UTF8 characters. If the utf8mb4 charset is not available, special characters such as emojis, less common Asian characters, various ancient phifi, or mathematical symbols will be replaced with . You can read more details about this topic in these FAQs.

(I’ve only seen it now, you couldn’t have these alerts on the homepage, which is the page I look at most often?)

7

Hi,

1608938301263 looks a bit much for the page transfer time (I think it is in milliseconds, so it’s 50 years). I have no Idea why such a high number would be recorded in the browser (Matomo just uses the normal browser timing APIs), but I created a github issue for it:

The weird characters are fine I think, it’s just the error message failing to print out the binary-stored data.

8

Today another error, its similar?

ERROR [2021-01-18 11:10:17] 1489003  Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=20&period=month&date=2021-01-01&format=json&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file.  For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log
ERROR [2021-01-18 11:10:17] 1489003  Error unserializing the following response from ?module=API&method=CoreAdminHome.archiveReports&idSite=20&period=month&date=2021-01-01&format=json&trigger=archivephp: ''
ERROR [2021-01-18 11:18:31] 1489003  2 total errors during this script execution, please investigate and try and fix these errors.
ERROR [2021-01-18 11:18:31] 1489003  Uncaught exception: /var/www/html/matomo2/core/CronArchive.php(611): 2 total errors during this script execution, please investigate and try and fix these errors.
2 total errors during this script execution, please investigate and try and fix these errors.


                                                                                                 
  [Exception]                                                                                    
  2 total errors during this script execution, please investigate and try and fix these errors.  
                                                                                                 


core:archive [--url="..."] [--skip-idsites[="..."]] [--skip-all-segments] [--force-idsites[="..."]] [--skip-segments-today] [--force-periods[="..."]] [--force-date-last-n="..."] [--force-date-range[="..."]] [--force-idsegments="..."] [--concurrent-requests-per-website[="..."]] [--concurrent-archivers[="..."]] [--disable-scheduled-tasks] [--accept-invalid-ssl-certificate] [--php-cli-options[="..."]] [--force-all-websites]
9

Lots of the paid support folks who we talk to at Matomo have been getting good performance with 2G or 8G of memory in the memory_limit setting of PHP CLI.

Many of their Matomo servers are monitoring sites with more than 1M hits per month. But I also get good results using 2G of PHP memory on my own hobby site, which gets 200 hits a month and is running on a $5/month host.

It’s a good way to quell the “unserialization has failed error” or “memory exhausted” error.

Please see several such settings, at this guide: https://matomo.org/faq/new-to-piwik/faq_137/

1 Like