Piwik 1.8.2 Monthly Stats Reset to Zero

I recently installed Piwik 1.8.2 (new install) on my server and setup tracking for 1200 sites via the API. I turned off the on-demand updating and setup the cron job. Now it seems that each time the cron job runs it deletes the weekly/monthly stats for most sites (a few work… maybe those with traffic since the last update?), but 99% of sites have all their summaries go to zero. The daily stats are fine. If I delete all the archive_ tables and force an update on all sites then everything looks good, until the next cron…

Interesting, we were supposed to have fixed this bug in the previous release - but maybe there’s still another bug!

Do you see this behavior all day long, or at the beginning of the day in the website’s timezones?

What are the site timezones, are they all the same?

I will probably need access to your server to debug further, or the DB to replicate the issue?
Also some time sinc eI’m booked for next 2 weeks

Hi Matt,

I ran the archive script last night around 11PM with the following parameters: --force-all-websites --force-all-periods=157784630. I deleted all the archive tables first. Everything looks good (for the monthly stats) after running the script. When I looked at the ‘all websites’ view this morning I noticed that a bunch were zero again. Right now, if I compare the sites with visits for today it exactly matches the sites that show monthly visits > 0 (just sorting by visits on the ‘all websites’ page).

All sites are in the same timezone (America/Toronto).

I’ll check again tomorrow, but if the past repeats itself then ‘all websites’ will show zero for every site (monthly view) until each site starts getting some visits.

Thanks
Justin

I’m not familiar with the Piwik source code, but if you can point me in the right direction (what file/module to look at) then I’ll take a stab at fixing it myself. I looked at archive.php, but it seems to just call the web service for getVisits.

I’m looking in the archive tables in the database. I compared a site that works with one that doesn’t. Both have daily and weekly numbers (period = 1,2) but the working site has numbers for the month (period 3) while the unworking site has no values for period 3 at all (no rows).

In the CRON can you please setup as per the doc: How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo
without force-websites and force-periods etc.

In this use case (default cron call) what is the observed behavior after 1 day?

This is how my cron is configured:

/usr/bin/php5 /home/piwik/public_html/misc/cron/archive.php --url=http://piwik.xxxxxxxx.ca/ > /dev/null

OK thanks for checking that.

Now, are you confident looking in the piwik_archive_numeric table for the record for the month period=3, that have been zeroed, and see what is there?

Because the script should not “delete” the data so I’m surprised it does not display… but it really should not delete the rows with period=3, so maybe it updates them … ? can you see anything interesting?

As of right now, at 7:10AM local time, there are 5654 entries with period = 3. The monthly stats actually look pretty good right now. I’ll report back in a few hours.

SELECT idsite, count(*) FROM piwik_archive_numeric_2012_07 where period = 3
group by idsite
order by idsite

As of right now (10:30AM), there are only 272 rows in piwik_archive_2012_07 (with period=3) and they correspond to only the 14 sites that have had visits today

Thank you for the useful report.

Can you confirm you are NOT using the “Delete old report” or “Delete old logs” feature in Settings > Privacy?

If you’re not using these features then the bug must come from this code: http://dev.piwik.org/trac/browser/trunk/core/ArchiveProcessing/Period.php#L353

Could you please send me the table dump with all records, then table dump with the missing records? I will investigate

That’s correct, both of those options are set to ‘No’.

Now that we are into August (the second month of using Piwik), I see that monthly totals for July are correct, but August is missing details for all sites without visits today.

I don’t know what files you want from the database, so exported piwik_archive_numeric_2012_08 for two sites (one with visits today and one without).

Ok thanks I found an issue, which I think will also fix your problem. Please upgrade to 1.8.3-B4 from: Index of /

and wait 1 day to see if the bug is fixed?

Thanks!

Thanks. I’ll report back tomorrow.

That appears to have worked! Thanks for the quick fix!

Justin

Great news thanks for your help isolating the bug!

Hi Matt,

Does this issue have a ticket number in Piwik’s bug tracking system ?

I would need it as its’ a bug we’ve noticed and tracked in our system, and as it appears to be fixed (I didn’t test it yet) I would need to link the piwik ticket number to our tracking system.

Thanks
Patrick

see Huge DB Disk space savings! Database purge should trigger on all archive_* tables at least once a day · Issue #3003 · matomo-org/matomo · GitHub