Hi,
since 2.8.3 ( I went straight to 2.8.3 from 2.7.x)
Archive is blowing the memory on my VPS. This has never been an issue before.
Anyone else experiencing this?
Hi llocally, how is your piwik setup?
We have a speed regression in some cases where the log importer is used, on large piwik servers. See this issue: Slow query from purgeInvalidatedArchiveTable (query is 3-10min long on bigger instances) · Issue #6508 · matomo-org/matomo · GitHub
Can you paste here an archive.log after the upgrade, and archive.log before the upgrade if you still have access?
Hmm, I don’t have logs as I overwrite by default. It is inconsistent, e.g. it last failed at 10 but the last two are running fine.
I manually ran one (based on the output below) and ran htop at the same time, the memory barely moved (950MB inuse out of 2GB) but cpu did and it returned data OK
(output token / domain altered of course for security )
Cron error report at the time the memory gets eaten. Obviously getting no output.
ERROR CoreConsole[2014-10-25 09:33:35] [10c32] Error unserializing the following response from https://analytics-01.mydomain.uk/index.php?module=API&method=API.get&idSite=170&period=month&date=last2&format=php&token_auth=bcc0a2b494121f953be5ed59c83ed6798a&trigger=archivephp: Got invalid response from API request: https://analytics-01.mydomain.uk/index.php?module=API&method=API.get&idSite=170&period=month&date=last2&format=php&token_auth=bcc0a2b494121f953be5ed59c83ed6798a&trigger=archivephp. Response was 'curl_exec: Empty reply from server. Hostname requested was: analytics-01.mydomain.uk’
ERROR CoreConsole[2014-10-25 09:35:14] [10c32] 1 total errors during this script execution, please investigate and try and fix these errors.
OK here is alog of one that blew the memory
I have extracted just the relevant site and surrounding (and changed some details for privacy)
As you can see, the site blowing memory has 4 times as many visits as the ones around, that worked.
Any thoughts?
INFO CoreConsole[2014-10-25 17:33:23] [c901a] Archived website id = 160, period = year, 7085 visits in last last2 years, 7085 visits this year, Time elapsed: 4.608s
INFO CoreConsole[2014-10-25 17:33:23] [c901a] Archived website id = 160, 4 API requests, Time elapsed: 15.210s [8/21 done]
INFO CoreConsole[2014-10-25 17:33:28] [c901a] Archived website id = 170, period = day, 1277 visits in last last2 days, 571 visits today, Time elapsed: 5.097s
INFO CoreConsole[2014-10-25 17:33:33] [c901a] Archived website id = 170, period = week, 10246 visits in last last2 weeks, 4690 visits this week, Time elapsed: 5.268s
INFO CoreConsole[2014-10-25 17:33:51] [c901a] Archived website id = 170, period = month, 39101 visits in last last2 months, 18403 visits this month, Time elapsed: 17.915s
ERROR CoreConsole[2014-10-25 17:34:02] [c901a] Error unserializing the following response from https://analytics-01.mydomain.uk/index.php?module=API&method=API.get&idSite=170&period=year&date=last2&format=php&token_auth=xx2b494121f953be5ed59c83ed6798a&trigger=archivephp: Got invalid response from API request: https://analytics-01.mydomain.uk/index.php?module=API&method=API.get&idSite=170&period=year&date=last2&format=php&token_auth=xx4121f953be5ed59c83ed6798a&trigger=archivephp. Response was 'curl_exec: Empty reply from server. Hostname requested was: analytics-01.mydomain.uk’
INFO CoreConsole[2014-10-25 17:34:02] [c901a] Archived website id = 170, period = year, 0 visits in last last2 years, 0 visits this year, Time elapsed: 10.641s
INFO CoreConsole[2014-10-25 17:34:02] [c901a] Archived website id = 170, 4 API requests, Time elapsed: 39.086s [9/21 done]
INFO CoreConsole[2014-10-25 17:34:06] [c901a] Archived website id = 182, period = day, 7 visits in last last2 days, 2 visits today, Time elapsed: 3.705s
INFO CoreConsole[2014-10-25 17:34:08] [c901a] Archived website id = 182, period = week, 64 visits in last last2 weeks, 24 visits this week, Time elapsed: 1.461s
INFO CoreConsole[2014-10-25 17:34:11] [c901a] Archived website id = 182, period = month, 223 visits in last last2 months, 109 visits this month, Time el
Hmm, I have Piwik version 2.8.3 running with only 128 MB memory for PHP. And I am currently processing 10.000.000+ loglines per day.
It looks like that the webserver has a problem. Maybe looking into the webserver log files for a (PHP) error. Or if you have PHP log files, look into them. I think that PHP throws an error, and the webserver can’t handle that.
I’ll look at the Apache/PHP logs.
I can say it doesn’t happen every run, but when it happens it eats memory in seconds.
My server is still on PHP 5.3 btw
ERROR CoreConsole[2014-10-27 08:33:44] [f6edf] Error unserializing the following response from https://analytics-01.mydomain.uk/index.php?module=API&method=API.get&idSite=170&period=month&date=last2&format=php&token_auth=b0a2www121f953be5ed59c83ed6798a&trigger=archivephp:
Occurs on the same site each time it fails
I’m wondering if it is possible corrupt data making the archive throw in to a memory eating loop?
I would just delete the site from piwik if it wasn’t quite an important one and the history useful.
Maybe I’ll do some MySQL checking!
I know that you can enable debug in the config.ini.php file in the Piwik/config folder. This will give a lot of debug, en I think also the queries that are fired.
Look in the config.global.ini.php for logging settings, and put them in the config.ini.php file. The last file overwrite the system defaults. Maybe if you enable more verbosity, we can find the problem.
I tried adding
[log]
logger_message[] = “screen”
logger_error[] = “screen”
logger_exception[] = “screen”
log_only_when_debug_parameter = 1
[Debug]
enable_sql_profiler = 1
But then found I couldn’t add the &debug to the archive url and it builds its own!
(BTW I checked the DB tables all OK)
Maybe I am wrong, but you can also enable logging for CLI. The archive process is in my case a CLI action. And you don’t have to manipulate the urls.
Couldn’t see it (a log / debug option) in the CLI help for core:archive