Hi!
I have noticed some strange behaviour in Matomo API.
Until a few days ago I had Matomo version 3.7.“don’t remember exactly”, where i had a problem of general slowness, both in the main report page and in accessing the settings and sections without report. When opening whatever page my server’s hard disk started to make noise.
I have not found solutions for this particular problem.
Then i tried the API. I’d like to use the API to retrieve daily and total visits for my websites, so i built these links:
matomo/?module=API&method=VisitsSummary.getVisits&idSite=1&period=day&date=today&format=JSON&token_auth=…
and
matomo/?module=API&method=VisitsSummary.getVisits&idSite=1&period=range&date=2018-01-01,today&format=JSON&token_auth=…
As I expected, even the API was very slow.
The strage thing is that once the API slowly loaded the results, if I do the same request, it will answer quickly, and if I do the same request within a minute, the same quick response arrived, like if it had a cache or something else.
After a minute, i had to wait for the same slow times.
Then Matomo 3.8.0 arrived. Right ater update I noticed a nearly istant loading times regarding the main interface, including reports and settings pages.
I tried again the API, but no luck, it did’t change anything.
Then i decided to do a fresh install of Matomo 3.8.0, to ensure the problem was not regarded to the 3.7.X version previous data.
Same fast behaviour of the main and settings pages and same slow behaviour of the visit count API.
Am I doing something wrong?
I’m on Windows Server 2012 R2, with the last versione of Apache, 2.4.38, the last versione of PHP, 7.3 and the last version of MySQL server 5.7 (because before Matomo fresh reinstallation, i reinstalled also all server-side softwares).
The previous version of Apache, PHP e MySQL were 2.4.35, 7.2.1 and 5.6.
I know very little about Windows, so take my tips with a grain of salt:
Seems like you are benefiting from one change in 3.8: Formerly Matomo used files for sessions, so on every request it would need to read a file on disk to know if you are logged in and which user you are. Now Matomo is saving this data in the database by default. As it is really small I guess your MySQL keeps it in Memory which makes loading UI pages (especially things like settings) much faster.
This of course doesn’t help with larger database queries needed to create reports or power the API.
It sounds like your disk is pretty slow (if a hard drive makes significant noise I wouldn’t trust it to live too long any more and start doing lots of backups) and probably therefore makes MySQL slow when doing larger queries.
One idea for improvement I have (that doesn’t involve adding an SSD or more RAM to avoid reading from disk) is setting up a cron job (if you haven’t already) and disable browser archiving. This won’t make creating reports faster, but they are already created when you open the browser (with the disadvantage of all reports being up to date for the last cron run).
If you notice that your tracking requests (so the calls to piwik.php) are also running really slowly, you could set up https://plugins.matomo.org/QueuedTracking (with MySQL or Redis). This allows the request to directly be stored raw and every x requests the query gets processed asynchronously.
Thank you very much for these tips!
I already tried the cron-job method with the older version of Matomo, but with no results.
I just tried again, but an error happens:
Error: ‘runCronArchiving’ not existing or not available in module ‘\Piwik\Plugins\CoreAdminHome\API’. (translated from Italian).
maybe I’ll keep Matomo just to analyze the traffic with the web interface and I will do otherwise for the API…
I think I have seen this error bevor. It happens when one of the early 3.x updates didn’t run properly and somehow the token_auth of the admin user got broken.
Could you try resetting the token_auth of your admin user (if you are using it somewhere, you also have to change it there) or maybe create a new superuser account and delete the original one.