Hi,
how can I confirm that reports have been archived and old logfiles can be deleted?
I have a cron running, also archiving should be done with generating reports in the browser. But I am not sure if the archiving really is executed.
Thanks for any help in advance!
Hi @bnetz ,
Are you storing archive logs in a separate file? From the log file you can check the archive job is being completed successfully or not.
5 * * * * www-data /usr/bin/php /path/to/matomo/console core:archive --url=http://example.org/matomo/ > /home/example/matomo-archive.log
For example, matomo-archive.log file will get the logs of archive job.
Thanks, @Naser_Aslam - yes, I store them separately, but I am not sure if this means 100% it was done successful:
O [2023-01-15 18:34:18] 726419 ---------------------------
INFO [2023-01-15 18:34:18] 726419 INIT
INFO [2023-01-15 18:34:18] 726419 Running Matomo 4.13.0 as Super User
INFO [2023-01-15 18:34:18] 726419 ---------------------------
INFO [2023-01-15 18:34:18] 726419 NOTES
INFO [2023-01-15 18:34:18] 726419 - If you execute this script at least once per hour (or more often) in a crontab, you may disable 'Browser trigger archiving' in Matomo UI > Settings > General Settings.
INFO [2023-01-15 18:34:18] 726419 See the doc at: https://matomo.org/docs/setup-auto-archiving/
INFO [2023-01-15 18:34:18] 726419 - Async process archiving supported, using CliMulti.
INFO [2023-01-15 18:34:18] 726419 - Reports for today will be processed at most every 3600 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2023-01-15 18:34:18] 726419 - Archiving was last executed without error 4 hours 19 min ago.
INFO [2023-01-15 18:34:18] 726419 ---------------------------
INFO [2023-01-15 18:34:18] 726419 START
INFO [2023-01-15 18:34:18] 726419 Starting Matomo reports archiving...
INFO [2023-01-15 18:34:18] 726419 Start processing archives for site 1.
INFO [2023-01-15 18:34:18] 726419 Will invalidate archived reports for today in site ID = 1's timezone (2023-01-15 00:00:00).
INFO [2023-01-15 18:34:18] 726419 LOAD DATA INFILE failed... Error was:
Try #1: LOAD DATA INFILE : SQLSTATE[28000]: Invalid authorization specification: 1045 Access denied for user '++++++++++++'@'%' (using password: YES)[28000],
Try #2: LOAD DATA LOCAL INFILE : SQLSTATE[42000]: Syntax error or access violation: 1148 The used command is not allowed with this MariaDB version[42000]
INFO [2023-01-15 18:34:18] 726419 Will invalidate archived reports for yesterday in site ID = 1's timezone (2023-01-14 00:00:00).
INFO [2023-01-15 18:34:20] 726419 Archived website id 1, period = day, date = 2023-01-15, segment = '', 15692 visits found. Time elapsed: 2.720s
INFO [2023-01-15 18:34:23] 726419 Archived website id 1, period = week, date = 2023-01-09, segment = '', 178451 visits found. Time elapsed: 2.735s
INFO [2023-01-15 18:34:26] 726419 Archived website id 1, period = month, date = 2023-01-01, segment = '', 267571 visits found. Time elapsed: 3.335s
INFO [2023-01-15 18:34:27] 726419 Archived website id 1, period = year, date = 2023-01-01, segment = '', 267571 visits found. Time elapsed: 0.668s
INFO [2023-01-15 18:34:27] 726419 Finished archiving for site 1, 4 API requests, Time elapsed: 9.574s [1 / 1 done]
INFO [2023-01-15 18:34:27] 726419 Done archiving!
INFO [2023-01-15 18:34:27] 726419 ---------------------------
INFO [2023-01-15 18:34:27] 726419 SUMMARY
INFO [2023-01-15 18:34:27] 726419 Processed 4 archives.
INFO [2023-01-15 18:34:27] 726419 Total API requests: 4
INFO [2023-01-15 18:34:27] 726419 done: 4 req, 9599 ms, no error
INFO [2023-01-15 18:34:27] 726419 Time elapsed: 9.600s
INFO [2023-01-15 18:34:27] 726419 ---------------------------
INFO [2023-01-15 18:34:27] 726419 SCHEDULED TASKS
INFO [2023-01-15 18:34:27] 726419 Starting Scheduled tasks...
INFO [2023-01-15 18:34:27] 726419 done
INFO [2023-01-15 18:34:27] 726419 ---------------------------
LOAD DATA INFILE would improve performance, but archiving is done successfully without it.
check here: How do I get LOAD DATA INFILE to work on my server? FAQ - Analytics Platform - Matomo
with those numbers of visits and that archiving duration you are fine without it i guess.
3 Likes
Hi @bnetz ,
Yes the archive job is being completed successfully, you can take a look at the following screenshot.
However, you are getting a warning. You can go with this warning but if you fix this issue it will be good for better performance of Matomo.
2 Likes
Thanks, @soardiac - I asked the hoster to fix this and he did, so everything works ok now!
@Naser_Aslam Thank you - indeed I got the error fixed with help of my hoster, now everything is ok.
1 Like
Related to "how do I tell if the archiver ran? How do I tell that Raw Data got turned into Report Data?
- In the Matomo interface watch for flatlines or too-low spots in your usual pattern.
- In the Matomo interface look for hollow dots. They are hollow to show that a “invalid” tag is lingering on that day/week/month/year. Every time core:archive does its archiving, it grabs a list of the invalid report times and should fix these, so they shouldn’t linger in this hollow state. If they do, it shows that the archiving for that period is crashing or somehow being skipped. Look up invalidate in the docs for some context.
- Daily or weekly check your error logs from archiving. The php console
core:archiveruns at the PHP-CLI module. To find where it logs its errors, checkphp -i | grep -i error. And your cron has error reporting in its output - look in your cron job definition.
1 Like