Hi,
We recently upgraded matomo from 3.x to 4.9.1, and we are trying to delete all data for a date range for a specific website, using the console, but the day counter in the period ‘day’ still shows visits.
For example 2022-05-01 until 2022-05-10 for the specific site id 31. This site has pageviews for every day, and is already fully archived for all periods. Then we try to delete this data.
First I deleted the logs:
console core:delete-logs-data --dates 20220501,20220511 --idsite 31
...
Then I invalidated the existing reports:
console core:invalidate-report-data --dates 2022-05-01,2022-05-10 --sites 31
INFO [2022-05-09 12:25:42] 1704137 Invalidating day periods in 2022-05-01,2022-05-10 [segment = ]...
INFO [2022-05-09 12:25:42] 1704137 Invalidating week periods in 2022-05-01,2022-05-10 [segment = ]...
INFO [2022-05-09 12:25:42] 1704137 Invalidating month periods in 2022-05-01,2022-05-10 [segment = ]...
INFO [2022-05-09 12:25:42] 1704137 Invalidating year periods in 2022-05-01,2022-05-10 [segment = ]...
And then I triggered the archive command to see the results in the reports:
console core:archive --force-date-range 2022-05-01,2022-05-10 --force-idsites 31
INFO [2022-05-09 12:28:28] 1707186 ---------------------------
INFO [2022-05-09 12:28:28] 1707186 INIT
INFO [2022-05-09 12:28:28] 1707186 Running Matomo 4.9.1 as Super User
INFO [2022-05-09 12:28:28] 1707186 ---------------------------
INFO [2022-05-09 12:28:28] 1707186 NOTES
INFO [2022-05-09 12:28:28] 1707186 - Async process archiving supported, using CliMulti.
INFO [2022-05-09 12:28:28] 1707186 - Reports for today will be processed at most every 36000 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2022-05-09 12:28:28] 1707186 - Archiving was last executed without error 15 Minuten 52s ago.
INFO [2022-05-09 12:28:28] 1707186 - Will process 1 websites (--force-idsites)
INFO [2022-05-09 12:28:28] 1707186 - Will process specified sites: 31
INFO [2022-05-09 12:28:28] 1707186 ---------------------------
INFO [2022-05-09 12:28:28] 1707186 START
INFO [2022-05-09 12:28:28] 1707186 Starting Matomo reports archiving...
INFO [2022-05-09 12:28:28] 1707186 Start processing archives for site 31.
INFO [2022-05-09 12:28:29] 1707186 Archived website id 31, period = week, date = 2022-05-09, segment = '', 0 visits found. Time elapsed: 0.310s
INFO [2022-05-09 12:28:29] 1707186 Archived website id 31, period = week, date = 2022-05-02, segment = '', 0 visits found. Time elapsed: 0.310s
INFO [2022-05-09 12:28:29] 1707186 Archived website id 31, period = day, date = 2022-05-01, segment = '', 4 visits found. Time elapsed: 0.600s
INFO [2022-05-09 12:28:29] 1707186 Archived website id 31, period = month, date = 2022-05-01, segment = '', 4 visits found. Time elapsed: 0.173s
INFO [2022-05-09 12:28:29] 1707186 Archived website id 31, period = week, date = 2022-04-25, segment = '', 1270 visits found. Time elapsed: 0.247s
INFO [2022-05-09 12:28:30] 1707186 Archived website id 31, period = year, date = 2022-01-01, segment = '', 27352 visits found. Time elapsed: 0.316s
INFO [2022-05-09 12:28:30] 1707186 Finished archiving for site 31, 6 API requests, Time elapsed: 1.480s [1 / 1 done]
INFO [2022-05-09 12:28:30] 1707186 Done archiving!
INFO [2022-05-09 12:28:30] 1707186 ---------------------------
INFO [2022-05-09 12:28:30] 1707186 SUMMARY
INFO [2022-05-09 12:28:30] 1707186 Processed 6 archives.
INFO [2022-05-09 12:28:30] 1707186 Total API requests: 6
INFO [2022-05-09 12:28:30] 1707186 done: 6 req, 1482 ms, no error
INFO [2022-05-09 12:28:30] 1707186 Time elapsed: 1.482s
INFO [2022-05-09 12:28:30] 1707186 ---------------------------
INFO [2022-05-09 12:28:30] 1707186 SCHEDULED TASKS
INFO [2022-05-09 12:28:30] 1707186 Starting Scheduled tasks...
INFO [2022-05-09 12:28:30] 1707186 done
INFO [2022-05-09 12:28:30] 1707186 ---------------------------
After this the weeks, months and years are showing 0 visits in the GUI, which is expected.
But the day periods are still there as before. So my question is how to delete these day reports to get 0 visits and actions for these days?
I tried it again using the -vvv flag, which shows “No useable archive exists” and “Found invalidated archive we can skip (no visits)”. What does that mean?
Full output:
console core:archive --force-date-range 2022-05-01,2022-05-10 --force-idsites 31 -vvv
INFO [2022-05-09 12:41:05] 1709901 ---------------------------
INFO [2022-05-09 12:41:05] 1709901 INIT
INFO [2022-05-09 12:41:05] 1709901 Running Matomo 4.9.1 as Super User
INFO [2022-05-09 12:41:05] 1709901 ---------------------------
INFO [2022-05-09 12:41:05] 1709901 NOTES
INFO [2022-05-09 12:41:05] 1709901 - Async process archiving supported, using CliMulti.
INFO [2022-05-09 12:41:05] 1709901 - Reports for today will be processed at most every 36000 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2022-05-09 12:41:05] 1709901 - Archiving was last executed without error 2 Minuten 46s ago.
INFO [2022-05-09 12:41:05] 1709901 - Will process 1 websites (--force-idsites)
INFO [2022-05-09 12:41:05] 1709901 - Will process specified sites: 31
INFO [2022-05-09 12:41:05] 1709901 ---------------------------
INFO [2022-05-09 12:41:05] 1709901 START
INFO [2022-05-09 12:41:05] 1709901 Starting Matomo reports archiving...
DEBUG [2022-05-09 12:41:05] 1709901 Applying queued rearchiving...
INFO [2022-05-09 12:41:05] 1709901 Start processing archives for site 31.
DEBUG [2022-05-09 12:41:05] 1709901 Checking for queued invalidations...
DEBUG [2022-05-09 12:41:05] 1709901 Today archive can be skipped due to no visits for idSite = 31, skipping invalidation...
DEBUG [2022-05-09 12:41:05] 1709901 Yesterday archive can be skipped due to no visits for idSite = 31, skipping invalidation...
DEBUG [2022-05-09 12:41:05] 1709901 Done invalidating
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2039, idsite = 31, period = day(2022-05-10 - 2022-05-10), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = 2022-05-09 05:04:00, now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2037, idsite = 31, period = day(2022-05-09 - 2022-05-09), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = 2022-05-09 12:28:29, now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Processing invalidation: [idinvalidation = 2038, idsite = 31, period = week(2022-05-09 - 2022-05-15), name = done, segment = ].
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2036, idsite = 31, period = day(2022-05-08 - 2022-05-08), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2035, idsite = 31, period = day(2022-05-07 - 2022-05-07), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2034, idsite = 31, period = day(2022-05-06 - 2022-05-06), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2033, idsite = 31, period = day(2022-05-05 - 2022-05-05), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2032, idsite = 31, period = day(2022-05-04 - 2022-05-04), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2031, idsite = 31, period = day(2022-05-03 - 2022-05-03), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Found invalidated archive we can skip (no visits): [idinvalidation = 2029, idsite = 31, period = day(2022-05-02 - 2022-05-02), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Processing invalidation: [idinvalidation = 2030, idsite = 31, period = week(2022-05-02 - 2022-05-08), name = done, segment = ].
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Processing invalidation: [idinvalidation = 2027, idsite = 31, period = day(2022-05-01 - 2022-05-01), name = done, segment = ].
DEBUG [2022-05-09 12:41:05] 1709901 Starting archiving for ?module=API&method=CoreAdminHome.archiveReports&idSite=31&period=week&date=2022-05-09&format=json&trigger=archivephp
DEBUG [2022-05-09 12:41:05] 1709901 Starting archiving for ?module=API&method=CoreAdminHome.archiveReports&idSite=31&period=week&date=2022-05-02&format=json&trigger=archivephp
DEBUG [2022-05-09 12:41:05] 1709901 Starting archiving for ?module=API&method=CoreAdminHome.archiveReports&idSite=31&period=day&date=2022-05-01&format=json&trigger=archivephp
DEBUG [2022-05-09 12:41:05] 1709901 Running command: /usr/bin/php -q /var/dvt/html/vweban02.srv.int.dvt.at/matomo/console climulti:request -q --matomo-domain='' --superuser 'module=API&method=CoreAdminHome.archiveReports&idSite=31&period=week&date=2022-05-09&format=json&trigger=archivephp&pid=a627e679709b82dbb98109cc0f8c85fab34faba54fc020a2d6d5b9a851aef51acb3871e0d39f886262bd683b59f4b008d0760&runid=1709901' > /var/dvt/html/vweban02.srv.int.dvt.at/matomo/tmp/climulti/a627e679709b82dbb98109cc0f8c85fab34faba54fc020a2d6d5b9a851aef51acb3871e0d39f886262bd683b59f4b008d0760.output 2>&1 &
DEBUG [2022-05-09 12:41:05] 1709901 Running command: /usr/bin/php -q /var/dvt/html/vweban02.srv.int.dvt.at/matomo/console climulti:request -q --matomo-domain='' --superuser 'module=API&method=CoreAdminHome.archiveReports&idSite=31&period=week&date=2022-05-02&format=json&trigger=archivephp&pid=28111357f59734ad2439833d94798e21e2a4e663461d42f0bddc8a6a52178c148a317a51d0baf1941ca0a371b640320cc2f01&runid=1709901' > /var/dvt/html/vweban02.srv.int.dvt.at/matomo/tmp/climulti/28111357f59734ad2439833d94798e21e2a4e663461d42f0bddc8a6a52178c148a317a51d0baf1941ca0a371b640320cc2f01.output 2>&1 &
DEBUG [2022-05-09 12:41:05] 1709901 Running command: /usr/bin/php -q /var/dvt/html/vweban02.srv.int.dvt.at/matomo/console climulti:request -q --matomo-domain='' --superuser 'module=API&method=CoreAdminHome.archiveReports&idSite=31&period=day&date=2022-05-01&format=json&trigger=archivephp&pid=ff0bfe43e06fbf13ae0fc4819776aaa88c5a4891af30da1529ab5309ea762d2112b09dc407ffcc011126072554a8dbe66a1a2&runid=1709901' > /var/dvt/html/vweban02.srv.int.dvt.at/matomo/tmp/climulti/ff0bfe43e06fbf13ae0fc4819776aaa88c5a4891af30da1529ab5309ea762d2112b09dc407ffcc011126072554a8dbe66a1a2.output 2>&1 &
INFO [2022-05-09 12:41:05] 1709901 Archived website id 31, period = week, date = 2022-05-09, segment = '', 0 visits found. Time elapsed: 0.304s
INFO [2022-05-09 12:41:05] 1709901 Archived website id 31, period = week, date = 2022-05-02, segment = '', 0 visits found. Time elapsed: 0.304s
INFO [2022-05-09 12:41:05] 1709901 Archived website id 31, period = day, date = 2022-05-01, segment = '', 4 visits found. Time elapsed: 0.567s
DEBUG [2022-05-09 12:41:05] 1709901 No usable archive exists (ts_archived of existing = 2022-05-09 12:28:29, now = 2022-05-09 12:41:05).
DEBUG [2022-05-09 12:41:05] 1709901 Processing invalidation: [idinvalidation = 2028, idsite = 31, period = month(2022-05-01 - 2022-05-31), name = done, segment = ].
DEBUG [2022-05-09 12:41:05] 1709901 Found archive with intersecting period with others in concurrent batch, skipping until next batch: [idinvalidation = 2042, idsite = 31, period = week(2022-04-25 - 2022-05-01), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 Found archive with intersecting period with others in concurrent batch, skipping until next batch: [idinvalidation = 2043, idsite = 31, period = year(2022-01-01 - 2022-12-31), name = done, segment = ]
DEBUG [2022-05-09 12:41:05] 1709901 No next invalidated archive.
DEBUG [2022-05-09 12:41:05] 1709901 Starting archiving for ?module=API&method=CoreAdminHome.archiveReports&idSite=31&period=month&date=2022-05-01&format=json&trigger=archivephp
DEBUG [2022-05-09 12:41:05] 1709901 Running command: /usr/bin/php -q /var/dvt/html/vweban02.srv.int.dvt.at/matomo/console climulti:request -q --matomo-domain='' --superuser 'module=API&method=CoreAdminHome.archiveReports&idSite=31&period=month&date=2022-05-01&format=json&trigger=archivephp' 2>&1
INFO [2022-05-09 12:41:06] 1709901 Archived website id 31, period = month, date = 2022-05-01, segment = '', 4 visits found. Time elapsed: 0.176s
DEBUG [2022-05-09 12:41:06] 1709901 No usable archive exists (ts_archived of existing = , now = 2022-05-09 12:41:06).
DEBUG [2022-05-09 12:41:06] 1709901 Processing invalidation: [idinvalidation = 2042, idsite = 31, period = week(2022-04-25 - 2022-05-01), name = done, segment = ].
DEBUG [2022-05-09 12:41:06] 1709901 Found archive with intersecting period with others in concurrent batch, skipping until next batch: [idinvalidation = 2043, idsite = 31, period = year(2022-01-01 - 2022-12-31), name = done, segment = ]
DEBUG [2022-05-09 12:41:06] 1709901 No next invalidated archive.
DEBUG [2022-05-09 12:41:06] 1709901 Starting archiving for ?module=API&method=CoreAdminHome.archiveReports&idSite=31&period=week&date=2022-04-25&format=json&trigger=archivephp
DEBUG [2022-05-09 12:41:06] 1709901 Running command: /usr/bin/php -q /var/dvt/html/vweban02.srv.int.dvt.at/matomo/console climulti:request -q --matomo-domain='' --superuser 'module=API&method=CoreAdminHome.archiveReports&idSite=31&period=week&date=2022-04-25&format=json&trigger=archivephp' 2>&1
INFO [2022-05-09 12:41:06] 1709901 Archived website id 31, period = week, date = 2022-04-25, segment = '', 1270 visits found. Time elapsed: 0.255s
DEBUG [2022-05-09 12:41:06] 1709901 No usable archive exists (ts_archived of existing = 2022-05-09 12:28:30, now = 2022-05-09 12:41:06).
DEBUG [2022-05-09 12:41:06] 1709901 Processing invalidation: [idinvalidation = 2043, idsite = 31, period = year(2022-01-01 - 2022-12-31), name = done, segment = ].
DEBUG [2022-05-09 12:41:06] 1709901 No next invalidated archive.
DEBUG [2022-05-09 12:41:06] 1709901 Starting archiving for ?module=API&method=CoreAdminHome.archiveReports&idSite=31&period=year&date=2022-01-01&format=json&trigger=archivephp
DEBUG [2022-05-09 12:41:06] 1709901 Running command: /usr/bin/php -q /var/dvt/html/vweban02.srv.int.dvt.at/matomo/console climulti:request -q --matomo-domain='' --superuser 'module=API&method=CoreAdminHome.archiveReports&idSite=31&period=year&date=2022-01-01&format=json&trigger=archivephp' 2>&1
INFO [2022-05-09 12:41:06] 1709901 Archived website id 31, period = year, date = 2022-01-01, segment = '', 27352 visits found. Time elapsed: 0.332s
DEBUG [2022-05-09 12:41:06] 1709901 No next invalidated archive.
INFO [2022-05-09 12:41:06] 1709901 Finished archiving for site 31, 6 API requests, Time elapsed: 1.479s [1 / 1 done]
DEBUG [2022-05-09 12:41:06] 1709901 No more sites left to archive, stopping.
INFO [2022-05-09 12:41:06] 1709901 Done archiving!
INFO [2022-05-09 12:41:06] 1709901 ---------------------------
INFO [2022-05-09 12:41:06] 1709901 SUMMARY
INFO [2022-05-09 12:41:06] 1709901 Processed 6 archives.
INFO [2022-05-09 12:41:06] 1709901 Total API requests: 6
INFO [2022-05-09 12:41:06] 1709901 done: 6 req, 1481 ms, no error
INFO [2022-05-09 12:41:06] 1709901 Time elapsed: 1.481s
INFO [2022-05-09 12:41:06] 1709901 ---------------------------
INFO [2022-05-09 12:41:06] 1709901 SCHEDULED TASKS
DEBUG [2022-05-09 12:41:06] 1709901 31 scheduled tasks loaded
INFO [2022-05-09 12:41:06] 1709901 Starting Scheduled tasks...
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 0:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 1:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 2:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 3:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 4:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 5:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 6:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 7:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 8:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 9:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 10:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 11:
DEBUG [2022-05-09 12:41:06] 1709901 Executing tasks with priority 12:
INFO [2022-05-09 12:41:06] 1709901 done
INFO [2022-05-09 12:41:06] 1709901 ---------------------------
Thanks for your help, any hint is appreciated.
Simon