Cronjob archiving does not count all visitos on all sites

Hello,

I’m using Piwik with 19 websites and therefore I’ve setup a cron job for the archiving. It runs every night, but does not archive every website correctly. I’ve attached two screenshots. One before I activated the online archiving and one after. Can you please help me and show me what is wrong with my cronjob?

Thanks!

Cronjob command:

php56 /home/www/piw/console core:archive --url=https://piw.MYDOMAIN.de

Cronjob output

INFO [2016-04-12 00:30:06] ---------------------------
INFO [2016-04-12 00:30:07] INIT
INFO [2016-04-12 00:30:07] Running Piwik 2.16.0 as Super User
INFO [2016-04-12 00:30:07] ---------------------------
INFO [2016-04-12 00:30:07] NOTES
INFO [2016-04-12 00:30:07] - Reports for today will be processed at most every 1800 seconds. You can change this value in Piwik UI > Settings > General Settings.
INFO [2016-04-12 00:30:07] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
INFO [2016-04-12 00:30:07] - Archiving was last executed without error 23 hours 57 min ago
INFO [2016-04-12 00:30:08] ---------------------------
INFO [2016-04-12 00:30:08] START
INFO [2016-04-12 00:30:08] Starting Piwik reports archiving…
INFO [2016-04-12 00:30:08] - no new tracking data for website id 1 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 1 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 2 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 2 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 3 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 3 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 4 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 4 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 5 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 5 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 6 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 6 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 7 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 7 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 8 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 8 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 9 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 9 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 10 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 10 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 11 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 11 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 12 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 12 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 14 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 14 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 15 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 15 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 17 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 17 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 18 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 18 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 19 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 19 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 21 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 21 as archiving is not needed
INFO [2016-04-12 00:30:08] - no new tracking data for website id 23 (between 2016-04-11 22:00:00 and 2016-04-12 01:30:08)
INFO [2016-04-12 00:30:08] Skipped website id 23 as archiving is not needed
INFO [2016-04-12 00:30:08] Done archiving!
INFO [2016-04-12 00:30:08] ---------------------------
INFO [2016-04-12 00:30:08] SUMMARY
INFO [2016-04-12 00:30:08] Total visits for today across archived websites: 0
INFO [2016-04-12 00:30:08] Archived today’s reports for 0 websites
INFO [2016-04-12 00:30:08] Archived week/month/year for 0 websites
INFO [2016-04-12 00:30:08] Skipped 19 websites
INFO [2016-04-12 00:30:08] - 19 skipped because no new visit since the last script execution
INFO [2016-04-12 00:30:08] - 0 skipped because existing daily reports are less than 1800 seconds old
INFO [2016-04-12 00:30:08] - 0 skipped because existing week/month/year periods reports are less than 3600 seconds old
INFO [2016-04-12 00:30:08] Total API requests: 0
INFO [2016-04-12 00:30:08] done: 0/19 0%, 0 vtoday, 0 wtoday, 0 wperiods, 0 req, 376 ms, no error
INFO [2016-04-12 00:30:08] Time elapsed: 0.377s
INFO [2016-04-12 00:30:08] ---------------------------
INFO [2016-04-12 00:30:08] SCHEDULED TASKS
INFO [2016-04-12 00:30:08] Starting Scheduled tasks…
INFO [2016-04-12 00:30:08] Scheduler: executing task Piwik\Plugins\CoreAdminHome\Tasks.purgeOutdatedArchives…
INFO [2016-04-12 00:30:08] Purging archives in 118 archive tables.
INFO [2016-04-12 00:30:15] Deleted 120 rows in archive tables (numeric + blob) for 2016-01-15.
INFO [2016-04-12 00:30:16] Deleted 344 rows in archive tables (numeric + blob) for 2016-04-15.
INFO [2016-04-12 00:30:16] Scheduler: finished. Time elapsed: 7.736s
INFO [2016-04-12 00:30:16] Scheduler: executing task Piwik\Plugins\ExamplePlugin\Tasks.myTask…
INFO [2016-04-12 00:30:16] Scheduler: finished. Time elapsed: 0.000s
INFO [2016-04-12 00:30:16] Scheduler: executing task Piwik\Plugins\UsersManager\Tasks.setUserDefaultReportPreference…
INFO [2016-04-12 00:30:16] Scheduler: finished. Time elapsed: 0.031s
INFO [2016-04-12 00:30:16] Scheduler: executing task Piwik\Plugins\CoreAdminHome\Tasks.purgeInvalidatedArchives…
INFO [2016-04-12 00:30:16] Scheduler: finished. Time elapsed: 0.010s
INFO [2016-04-12 00:30:16] Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.deleteReportData…
INFO [2016-04-12 00:30:19] Scheduler: finished. Time elapsed: 3.285s
INFO [2016-04-12 00:30:19] Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.deleteLogData…
INFO [2016-04-12 00:30:26] Scheduler: finished. Time elapsed: 6.749s
INFO [2016-04-12 00:30:26] Scheduler: executing task Piwik\Plugins\CorePluginsAdmin\Tasks.clearAllCacheEntries…
INFO [2016-04-12 00:30:26] Scheduler: finished. Time elapsed: 0.017s
INFO [2016-04-12 00:30:26] Scheduler: executing task Piwik\Plugins\CorePluginsAdmin\Tasks.sendNotificationIfUpdatesAvailable…
INFO [2016-04-12 00:30:27] Scheduler: finished. Time elapsed: 1.134s
INFO [2016-04-12 00:30:27] Scheduler: executing task Piwik\Plugins\CoreAdminHome\Tasks.optimizeArchiveTable…
INFO [2016-04-12 00:30:28] Scheduler: finished. Time elapsed: 1.136s
INFO [2016-04-12 00:30:28] Scheduler: executing task Piwik\Plugins\CoreUpdater\Tasks.sendNotificationIfUpdateAvailable…
INFO [2016-04-12 00:30:29] Scheduler: finished. Time elapsed: 0.304s
INFO [2016-04-12 00:30:29] done
INFO [2016-04-12 00:30:29] ---------------------------

Dashboard with cronjob and online processing:

Are you using Piwik 2.16.1? Maybe you were affected by this one: Some periods may not be archived at all · Issue #9468 · matomo-org/piwik · GitHub

1 Like

Thanks, the mentioned workaround by adding --force-all-websites to the cronjob solved my problem.