Every hour running my archiving cron job I get a selection of errors like at the bottom of this page. I’ve changed these values in global.ini.php
minimum_memory_limit = -1
minimum_memory_limit_when_archiving = -1
And set the memory_limit to 8G in my php settings, confirmed on the system check screen.
Is there a way to log further why these API requests are timing out? Is there something I can do to get more information?
Thanks for your guidance.
Extra info: PHP 7.4.5. Matamo 3.13.4
INFO [2020-04-22 15:45:17] 156894 ---------------------------
INFO [2020-04-22 15:45:17] 156894 INIT
INFO [2020-04-22 15:45:17] 156894 Running Matomo 3.13.4 as Super User
INFO [2020-04-22 15:45:17] 156894 ---------------------------
INFO [2020-04-22 15:45:17] 156894 NOTES
INFO [2020-04-22 15:45:19] 156894 - Async process archiving supported, using CliMulti.
INFO [2020-04-22 15:45:19] 156894 - Reports for today will be processed at most every 3600 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-04-22 15:45:19] 156894 - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2020-04-22 15:45:19] 156894 - Archiving was last executed without error 17 hours 18 min ago
INFO [2020-04-22 15:45:19] 156894 - Will process 58 other websites because the last time they were archived was on a different day (in the website's timezone) , IDs: 3, 4, 5, 6, 7, 8, 9, 10, 12, 13, 16, 17, 19, 20, 22, 23, 25, 27, 28, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 42, 43, 44, 45, 46, 47, 48, 49, 50, 53, 54, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 73
INFO [2020-04-22 15:45:19] 156894 - Will process 1 other websites because some old data reports have been invalidated (eg. using the Log Import script or the InvalidateReports plugin) , IDs: 41
INFO [2020-04-22 15:45:22] 156894 ---------------------------
INFO [2020-04-22 15:45:22] 156894 START
INFO [2020-04-22 15:45:22] 156894 Starting Matomo reports archiving...
// SKIPPED A LOAD OF SUCCESSES
INFO [2020-04-22 15:49:42] 156894 Day has finished for website id 20 since last run
INFO [2020-04-22 15:49:42] 156894 Will pre-process for website id = 20, period = day, date = last2
INFO [2020-04-22 15:49:42] 156894 - pre-processing all visits
INFO [2020-04-22 15:49:45] 156894 Archived website id = 20, period = day, 0 segments, 60 visits in last 2 days, 33 visits today, Time elapsed: 2.672s
INFO [2020-04-22 15:49:46] 156894 Will pre-process for website id = 20, period = week, date = last2
INFO [2020-04-22 15:49:46] 156894 - pre-processing all visits
INFO [2020-04-22 15:49:48] 156894 Archived website id = 20, period = week, 0 segments, 300 visits in last 2 weeks, 77 visits this week, Time elapsed: 3.042s
INFO [2020-04-22 15:49:49] 156894 Will pre-process for website id = 20, period = month, date = last2
INFO [2020-04-22 15:49:49] 156894 - pre-processing all visits
INFO [2020-04-22 15:49:52] 156894 Archived website id = 20, period = month, 0 segments, 2755 visits in last 2 months, 1193 visits this month, Time elapsed: 4.490s
INFO [2020-04-22 15:49:54] 156894 Will pre-process for website id = 20, period = year, date = last2
INFO [2020-04-22 15:49:54] 156894 - pre-processing all visits
INFO [2020-04-22 15:49:56] 156894 Archived website id = 20, period = year, 0 segments, 7828 visits in last 2 years, 3619 visits this year, Time elapsed: 3.875s
INFO [2020-04-22 15:49:57] 156894 Will pre-process for website id = 20, period = range, date = last12
INFO [2020-04-22 15:49:57] 156894 - pre-processing all visits
INFO [2020-04-22 15:49:58] 156894 Archived website id = 20, period = range, 0 segments, 0 visits in last 30 ranges, 0 visits this range, Time elapsed: 2.102s
INFO [2020-04-22 15:49:58] 156894 Archived website id = 20, 5 API requests, Time elapsed: 16.188s [14/58 done]
INFO [2020-04-22 15:49:58] 156894 Day has finished for website id 22 since last run
INFO [2020-04-22 15:49:59] 156894 Will pre-process for website id = 22, period = day, date = last2
INFO [2020-04-22 15:49:59] 156894 - pre-processing all visits
INFO [2020-04-22 15:50:01] 156894 Archived website id = 22, period = day, 0 segments, 11 visits in last 2 days, 5 visits today, Time elapsed: 2.165s
INFO [2020-04-22 15:50:03] 156894 Will pre-process for website id = 22, period = week, date = last2
INFO [2020-04-22 15:50:03] 156894 - pre-processing all visits
INFO [2020-04-22 15:50:30] 156894 Archived website id = 22, period = week, 0 segments, 54 visits in last 2 weeks, 21 visits this week, Time elapsed: 29.630s
INFO [2020-04-22 15:50:38] 156894 Will pre-process for website id = 22, period = month, date = last2
INFO [2020-04-22 15:50:38] 156894 - pre-processing all visits
INFO [2020-04-22 15:50:48] 156894 Archived website id = 22, period = month, 0 segments, 0 visits in last 2 months, 0 visits this month, Time elapsed: 18.314s
INFO [2020-04-22 15:50:48] 156894 Archived website id = 22, 3 API requests, Time elapsed: 50.113s [15/58 done]
// SKIPPED A LOAD OF SUCCESSES
INFO [2020-04-22 15:59:38] 156894 Day has finished for website id 56 since last run
INFO [2020-04-22 15:59:38] 156894 Will pre-process for website id = 56, period = day, date = last2
INFO [2020-04-22 15:59:38] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:39] 156894 Archived website id = 56, period = day, 0 segments, 0 visits in last 2 days, 0 visits today, Time elapsed: 1.696s
INFO [2020-04-22 15:59:40] 156894 Will pre-process for website id = 56, period = week, date = last2
INFO [2020-04-22 15:59:40] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:42] 156894 Archived website id = 56, period = week, 0 segments, 0 visits in last 2 weeks, 0 visits this week, Time elapsed: 2.449s
INFO [2020-04-22 15:59:43] 156894 Will pre-process for website id = 56, period = month, date = last2
INFO [2020-04-22 15:59:43] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:44] 156894 Archived website id = 56, period = month, 0 segments, 0 visits in last 2 months, 0 visits this month, Time elapsed: 2.586s
INFO [2020-04-22 15:59:46] 156894 Will pre-process for website id = 56, period = year, date = last2
INFO [2020-04-22 15:59:46] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:48] 156894 Archived website id = 56, period = year, 0 segments, 5026 visits in last 2 years, 641 visits this year, Time elapsed: 3.371s
INFO [2020-04-22 15:59:49] 156894 Will pre-process for website id = 56, period = range, date = last12
INFO [2020-04-22 15:59:49] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:50] 156894 Archived website id = 56, period = range, 0 segments, 0 visits in last 30 ranges, 0 visits this range, Time elapsed: 2.032s
INFO [2020-04-22 15:59:50] 156894 Archived website id = 56, 5 API requests, Time elapsed: 12.146s [42/58 done]
INFO [2020-04-22 15:59:50] 156894 Day has finished for website id 57 since last run
INFO [2020-04-22 15:59:50] 156894 Will pre-process for website id = 57, period = day, date = last2
INFO [2020-04-22 15:59:50] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:52] 156894 Archived website id = 57, period = day, 0 segments, 12 visits in last 2 days, 4 visits today, Time elapsed: 2.144s
INFO [2020-04-22 15:59:53] 156894 Will pre-process for website id = 57, period = week, date = last2
INFO [2020-04-22 15:59:53] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:55] 156894 Archived website id = 57, period = week, 0 segments, 107 visits in last 2 weeks, 17 visits this week, Time elapsed: 3.219s
INFO [2020-04-22 15:59:57] 156894 Will pre-process for website id = 57, period = month, date = last2
INFO [2020-04-22 15:59:57] 156894 - pre-processing all visits
INFO [2020-04-22 15:59:59] 156894 Archived website id = 57, period = month, 0 segments, 422 visits in last 2 months, 267 visits this month, Time elapsed: 3.561s
INFO [2020-04-22 16:00:00] 156894 Will pre-process for website id = 57, period = year, date = last2
INFO [2020-04-22 16:00:00] 156894 - pre-processing all visits
INFO [2020-04-22 16:00:05] 156894 Archived website id = 57, period = year, 0 segments, 1456 visits in last 2 years, 517 visits this year, Time elapsed: 6.408s
INFO [2020-04-22 16:00:15] 156894 Will pre-process for website id = 57, period = range, date = last12
INFO [2020-04-22 16:00:15] 156894 - pre-processing all visits
INFO [2020-04-22 16:00:32] 156894 Archived website id = 57, period = range, 0 segments, 0 visits in last 30 ranges, 0 visits this range, Time elapsed: 26.835s
INFO [2020-04-22 16:00:32] 156894 Archived website id = 57, 5 API requests, Time elapsed: 42.179s [43/58 done]
INFO [2020-04-22 16:00:35] 156894 Day has finished for website id 58 since last run
INFO [2020-04-22 16:00:37] 156894 Will pre-process for website id = 58, period = day, date = last2
INFO [2020-04-22 16:00:37] 156894 - pre-processing all visits
INFO [2020-04-22 16:00:52] 156894 Day has finished for website id 59 since last run
INFO [2020-04-22 16:00:54] 156894 Will pre-process for website id = 59, period = day, date = last2
INFO [2020-04-22 16:00:54] 156894 - pre-processing all visits
INFO [2020-04-22 16:01:11] 156894 Day has finished for website id 60 since last run
INFO [2020-04-22 16:01:14] 156894 Will pre-process for website id = 60, period = day, date = last2
INFO [2020-04-22 16:01:14] 156894 - pre-processing all visits
INFO [2020-04-22 16:01:29] 156894 Day has finished for website id 61 since last run
INFO [2020-04-22 16:01:31] 156894 Will pre-process for website id = 61, period = day, date = last2
INFO [2020-04-22 16:01:31] 156894 - pre-processing all visits
INFO [2020-04-22 16:01:44] 156894 Day has finished for website id 62 since last run
INFO [2020-04-22 16:01:45] 156894 Will pre-process for website id = 62, period = day, date = last2
INFO [2020-04-22 16:01:45] 156894 - pre-processing all visits
INFO [2020-04-22 16:01:54] 156894 Archived website id = 62, period = day, 0 segments, 39 visits in last 2 days, 19 visits today, Time elapsed: 10.488s
INFO [2020-04-22 16:01:58] 156894 Will pre-process for website id = 62, period = week, date = last2
INFO [2020-04-22 16:01:58] 156894 - pre-processing all visits
INFO [2020-04-22 16:02:02] 156894 Archived website id = 62, period = week, 0 segments, 226 visits in last 2 weeks, 53 visits this week, Time elapsed: 7.710s
INFO [2020-04-22 16:02:11] 156894 Will pre-process for website id = 62, period = month, date = last2
INFO [2020-04-22 16:02:11] 156894 - pre-processing all visits
INFO [2020-04-22 16:02:27] 156894 Archived website id = 62, period = month, 0 segments, 1280 visits in last 2 months, 494 visits this month, Time elapsed: 25.001s
INFO [2020-04-22 16:02:31] 156894 Will pre-process for website id = 62, period = year, date = last2
INFO [2020-04-22 16:02:31] 156894 - pre-processing all visits
INFO [2020-04-22 16:02:36] 156894 Archived website id = 62, period = year, 0 segments, 13241 visits in last 2 years, 3089 visits this year, Time elapsed: 9.535s
INFO [2020-04-22 16:02:39] 156894 Will pre-process for website id = 62, period = range, date = last12
INFO [2020-04-22 16:02:39] 156894 - pre-processing all visits
INFO [2020-04-22 16:02:42] 156894 Archived website id = 62, period = range, 0 segments, 0 visits in last 30 ranges, 0 visits this range, Time elapsed: 5.236s
INFO [2020-04-22 16:02:42] 156894 Archived website id = 62, 5 API requests, Time elapsed: 57.978s [48/58 done]
INFO [2020-04-22 16:02:42] 156894 Day has finished for website id 63 since last run
INFO [2020-04-22 16:02:43] 156894 Will pre-process for website id = 63, period = day, date = last2
INFO [2020-04-22 16:02:43] 156894 - pre-processing all visits
INFO [2020-04-22 16:02:47] 156894 Archived website id = 63, period = day, 0 segments, 33 visits in last 2 days, 9 visits today, Time elapsed: 4.398s
INFO [2020-04-22 16:02:49] 156894 Will pre-process for website id = 63, period = week, date = last2
INFO [2020-04-22 16:02:49] 156894 - pre-processing all visits
INFO [2020-04-22 16:02:52] 156894 Archived website id = 63, period = week, 0 segments, 100 visits in last 2 weeks, 42 visits this week, Time elapsed: 5.517s
INFO [2020-04-22 16:02:55] 156894 Will pre-process for website id = 63, period = month, date = last2
INFO [2020-04-22 16:02:55] 156894 - pre-processing all visits
INFO [2020-04-22 16:02:58] 156894 Archived website id = 63, period = month, 0 segments, 549 visits in last 2 months, 240 visits this month, Time elapsed: 6.020s
INFO [2020-04-22 16:03:01] 156894 Will pre-process for website id = 63, period = year, date = last2
INFO [2020-04-22 16:03:01] 156894 - pre-processing all visits
INFO [2020-04-22 16:03:11] 156894 Archived website id = 63, period = year, 0 segments, 3936 visits in last 2 years, 994 visits this year, Time elapsed: 12.837s
INFO [2020-04-22 16:03:14] 156894 Will pre-process for website id = 63, period = range, date = last12
INFO [2020-04-22 16:03:14] 156894 - pre-processing all visits
INFO [2020-04-22 16:03:16] 156894 Archived website id = 63, period = range, 0 segments, 0 visits in last 30 ranges, 0 visits this range, Time elapsed: 4.965s
INFO [2020-04-22 16:03:16] 156894 Archived website id = 63, 5 API requests, Time elapsed: 33.742s [49/58 done]
INFO [2020-04-22 16:03:16] 156894 Day has finished for website id 64 since last run
INFO [2020-04-22 16:03:17] 156894 Will pre-process for website id = 64, period = day, date = last2
INFO [2020-04-22 16:03:17] 156894 - pre-processing all visits
INFO [2020-04-22 16:03:20] 156894 Archived website id = 64, period = day, 0 segments, 2 visits in last 2 days, 1 visits today, Time elapsed: 3.815s
INFO [2020-04-22 16:03:22] 156894 Will pre-process for website id = 64, period = week, date = last2
INFO [2020-04-22 16:03:22] 156894 - pre-processing all visits
INFO [2020-04-22 16:03:24] 156894 Archived website id = 64, period = week, 0 segments, 11 visits in last 2 weeks, 3 visits this week, Time elapsed: 4.142s
INFO [2020-04-22 16:03:26] 156894 Will pre-process for website id = 64, period = month, date = last2
INFO [2020-04-22 16:03:26] 156894 - pre-processing all visits
INFO [2020-04-22 16:03:29] 156894 Archived website id = 64, period = month, 0 segments, 68 visits in last 2 months, 33 visits this month, Time elapsed: 4.359s
INFO [2020-04-22 16:03:30] 156894 Will pre-process for website id = 64, period = year, date = last2
INFO [2020-04-22 16:03:30] 156894 - pre-processing all visits
INFO [2020-04-22 16:03:33] 156894 Archived website id = 64, period = year, 0 segments, 3650 visits in last 2 years, 259 visits this year, Time elapsed: 4.327s
INFO [2020-04-22 16:03:34] 156894 Will pre-process for website id = 64, period = range, date = last12
INFO [2020-04-22 16:03:34] 156894 - pre-processing all visits
INFO [2020-04-22 16:03:36] 156894 Archived website id = 64, period = range, 0 segments, 0 visits in last 30 ranges, 0 visits this range, Time elapsed: 2.477s
INFO [2020-04-22 16:03:36] 156894 Archived website id = 64, 5 API requests, Time elapsed: 19.127s [50/58 done]
// SKIPPED A LOAD OF SUCCESSES
INFO [2020-04-22 16:06:32] 156894 Done archiving!
INFO [2020-04-22 16:06:32] 156894 ---------------------------
INFO [2020-04-22 16:06:32] 156894 SUMMARY
INFO [2020-04-22 16:06:32] 156894 Total visits for today across archived websites: 503
INFO [2020-04-22 16:06:32] 156894 Archived today's reports for 54 websites
INFO [2020-04-22 16:06:32] 156894 Archived week/month/year for 54 websites
INFO [2020-04-22 16:06:32] 156894 Skipped 4 websites
INFO [2020-04-22 16:06:32] 156894 - 0 skipped because no new visit since the last script execution
INFO [2020-04-22 16:06:32] 156894 - 0 skipped because existing daily reports are less than 3600 seconds old
INFO [2020-04-22 16:06:32] 156894 - 0 skipped because existing week/month/year periods reports are less than 3600 seconds old
INFO [2020-04-22 16:06:32] 156894 - 4 skipped because got an error while querying reporting API
INFO [2020-04-22 16:06:32] 156894 Total API requests: 268
INFO [2020-04-22 16:06:32] 156894 done: 54/58 93%, 503 vtoday, 54 wtoday, 54 wperiods, 268 req, 1270715 ms, 10 errors.
INFO [2020-04-22 16:06:32] 156894 Time elapsed: 1270.716s
INFO [2020-04-22 16:06:32] 156894 ---------------------------
INFO [2020-04-22 16:06:32] 156894 SCHEDULED TASKS
INFO [2020-04-22 16:06:32] 156894 Starting Scheduled tasks...
INFO [2020-04-22 16:06:32] 156894 Scheduler: executing task Piwik\Plugins\CustomPiwikJs\Tasks.updateTracker...
INFO [2020-04-22 16:06:32] 156894 Scheduler: finished. Time elapsed: 0.007s
INFO [2020-04-22 16:06:32] 156894 Scheduler: executing task Piwik\Plugins\TagManager\Tasks.regenerateReleasedContainers...
INFO [2020-04-22 16:06:33] 156894 Scheduler: finished. Time elapsed: 0.232s
INFO [2020-04-22 16:06:33] 156894 Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.deleteLogData...
INFO [2020-04-22 16:06:33] 156894 Scheduler: finished. Time elapsed: 0.011s
INFO [2020-04-22 16:06:33] 156894 Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.anonymizePastData...
INFO [2020-04-22 16:06:33] 156894 Scheduler: finished. Time elapsed: 0.002s
INFO [2020-04-22 16:06:33] 156894 Scheduler: executing task Piwik\Plugins\UserCountry\GeoIPAutoUpdater.update...
INFO [2020-04-22 16:06:33] 156894 GeoLite databases have been discontinued. Skipping download of http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz. Consider switching to GeoIP 2.
INFO [2020-04-22 16:06:33] 156894 Scheduler: finished. Time elapsed: 0.018s
INFO [2020-04-22 16:06:33] 156894 done
INFO [2020-04-22 16:06:33] 156894 ---------------------------
INFO [2020-04-22 16:06:33] 156894 ---------------------------
INFO [2020-04-22 16:06:33] 156894 SUMMARY OF ERRORS
INFO [2020-04-22 16:06:33] 156894 Error: Got invalid response from API request: ?module=API&method=API.get&idSite=22&period=month&date=last2&format=php&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log
INFO [2020-04-22 16:06:33] 156894 Error: Got invalid response from API request: ?module=API&method=API.get&idSite=57&period=day&date=today&format=php&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log
INFO [2020-04-22 16:06:33] 156894 Error: Got invalid response from API request: ?module=API&method=API.get&idSite=58&period=day&date=last2&format=php&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log
INFO [2020-04-22 16:06:33] 156894 Error: Empty or invalid response '' for website id 58, Time elapsed: 15.498s, skipping
INFO [2020-04-22 16:06:33] 156894 Error: Got invalid response from API request: ?module=API&method=API.get&idSite=59&period=day&date=last2&format=php&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log
INFO [2020-04-22 16:06:33] 156894 Error: Empty or invalid response '' for website id 59, Time elapsed: 15.433s, skipping
INFO [2020-04-22 16:06:33] 156894 Error: Got invalid response from API request: ?module=API&method=API.get&idSite=60&period=day&date=last2&format=php&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log
INFO [2020-04-22 16:06:33] 156894 Error: Empty or invalid response '' for website id 60, Time elapsed: 15.663s, skipping
INFO [2020-04-22 16:06:33] 156894 Error: Got invalid response from API request: ?module=API&method=API.get&idSite=61&period=day&date=last2&format=php&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file. For more information and the error message please check in your PHP CLI error log file. As this core:archive command triggers PHP processes over the CLI, you can find where PHP CLI logs are stored by running this command: php -i | grep error_log
INFO [2020-04-22 16:06:33] 156894 Error: Empty or invalid response '' for website id 61, Time elapsed: 13.508s, skipping