Core:archive works on week, month and year but not on day


(Lucian Duma) #1

Hello,

Our piwik instance is feed with traffic data manually every morning using the Tracking API.

For this we thought of disabling the setting ‘Archive reports when viewed from the browser’.We’ve set up instead a daily windows scheduled task that calls the core:archive command.

For some time now (even before upgrading to version 2.15.0) we had problems with it: the command executed successfully, but the data was not up-to-date in the Dashboard.

When investigating, running the invalidate command and then the core:archive command manually shows some strange outputs:

  1. First command should invalidate all the data in the week 1-8 Nov 2015 for site 45:
    c:\wamp\bin\php\php5.4.3\php.exe C:\wamp\www\3SAnalytics\console core:invalidate-report-data --dates=2015-11-01,2011-11-08 --sites=45
    Output (probably OK):
    Invalidating day periods in 2015-11-01,2011-11-08 [segment = ]... Invalidating week periods in 2015-11-01,2011-11-08 [segment = ]... Invalidating month periods in 2015-11-01,2011-11-08 [segment = ]... Invalidating year periods in 2015-11-01,2011-11-08 [segment = ]...

  2. Second command should recompute the day, week, month and years periods:
    c:\wamp\bin\php\php5.4.3\php.exe C:\wamp\www\3SAnalytics\console core:archive --force-date-range=2015-11-01,2015-11-08 --url=http://3sAnalytics/ --force-idsites=45
    Output:
    INFO [2015-11-19 10:46:15] ---------------------------
    INFO [2015-11-19 10:46:15] INIT
    INFO [2015-11-19 10:46:15] Running Piwik 2.15.0 as Super User
    INFO [2015-11-19 10:46:15] ---------------------------
    INFO [2015-11-19 10:46:15] NOTES
    INFO [2015-11-19 10:46:15] - Reports for today will be processed at most every 10 seconds. You can change this value in Piwik UI > Settings > General Settings.
    INFO [2015-11-19 10:46:15] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
    INFO [2015-11-19 10:46:15] - Archiving was last executed without error 2 min 3s ago
    INFO [2015-11-19 10:46:15] - Will process 1 websites (–force-idsites)
    INFO [2015-11-19 10:46:15] ---------------------------
    INFO [2015-11-19 10:46:15] START
    INFO [2015-11-19 10:46:15] Starting Piwik reports archiving…
    INFO [2015-11-19 10:46:15] Will pre-process for website id = 45, period = day, date = 2015-11-01,2015-11-08
    INFO [2015-11-19 10:46:15] - pre-processing all visits
    INFO [2015-11-19 10:46:16] Archived website id = 45, period = day, 0 segments, 0 visits in days included in: 2015-11-01, 2015-11-08, Time elapsed: 1.440s
    INFO [2015-11-19 10:46:16] Will pre-process for website id = 45, period = week, date = 2015-11-01,2015-11-08
    INFO [2015-11-19 10:46:16] - pre-processing all visits
    INFO [2015-11-19 10:46:18] Archived website id = 45, period = week, 0 segments, 162 visits in weeks included in: 2015-11-01,2015-11-08, Time elapsed: 1.392s
    INFO [2015-11-19 10:46:18] Will pre-process for website id = 45, period = month, date = 2015-11-01,2015-11-08
    INFO [2015-11-19 10:46:18] - pre-processing all visits
    INFO [2015-11-19 10:46:22] Archived website id = 45, period = month, 0 segments, 149 visits in months included in: 2015-11-01,2015-11-08, Time elapsed: 4.083s
    INFO [2015-11-19 10:46:22] Will pre-process for website id = 45, period = year, date = 2015-11-01,2015-11-08
    INFO [2015-11-19 10:46:22] - pre-processing all visits
    INFO [2015-11-19 10:46:26] Archived website id = 45, period = year, 0 segments, 2049 visits in years included in: 2015-11-01,2015-11-08, Time elapsed: 4.119s
    INFO [2015-11-19 10:46:26] Archived website id = 45, 4 API requests, Time elapsed: 11.073s [1/1 done]
    INFO [2015-11-19 10:46:26] Done archiving!
    INFO [2015-11-19 10:46:26] ---------------------------
    INFO [2015-11-19 10:46:26] SUMMARY
    INFO [2015-11-19 10:46:26] Total visits for today across archived websites: 0
    INFO [2015-11-19 10:46:26] Archived today’s reports for 1 websites
    INFO [2015-11-19 10:46:26] Archived week/month/year for 1 websites
    INFO [2015-11-19 10:46:26] Skipped 35 websites
    INFO [2015-11-19 10:46:26] - 0 skipped because no new visit since the last script execution
    INFO [2015-11-19 10:46:26] - 0 skipped because existing daily reports are less than 10 seconds old
    INFO [2015-11-19 10:46:26] - 0 skipped because existing week/month/year periods reports are less than 3600 seconds old
    INFO [2015-11-19 10:46:26] Total API requests: 4
    INFO [2015-11-19 10:46:26] done: 1/1 100%, 0 vtoday, 1 wtoday, 1 wperiods, 4 req, 11100 ms, no error
    INFO [2015-11-19 10:46:26] Time elapsed: 11.101s
    INFO [2015-11-19 10:46:26] ---------------------------
    INFO [2015-11-19 10:46:26] SCHEDULED TASKS
    INFO [2015-11-19 10:46:26] Starting Scheduled tasks…
    INFO [2015-11-19 10:46:26] done
    INFO [2015-11-19 10:46:26] ---------------------------

What’s strange is this: 0 visits in days, 162 visits in weeks, 149 visits in months, 2049 visits in years.

I’ve used the following query to check if there is any data for the given period:
select * from piwik.piwik_log_visit as log inner join piwik.piwik_log_link_visit_action as log_act on log.idvisit = log_act.idvisit where log_act.server_time between '2015-11-01 00:00:000' and '2015-11-02 23:59:59' and log_act.idsite = 45 and log.idsite=45
and I’ve got over 1000 lines of traffic data.

Do you have any idea if this is normal or a bug, and how to fix this?

Thanks.
Lucian