Archiving problems

I’m using piwik with these stats: 11 sites, 1.7k daily visits, 7k daily actions. Truncating after 180 days, this results in 1.4M lines in piwik_log_link_visit_action and 300k lines in piwik_log_visit.

I have an issue with the achive.php script: because the memory is limited to 128M, I the monthly stat fails for two websites.

Since 14 days, when I updated to piwik 1.8, I have an additional issue: My piwik_archive_blob_2011_01 is increasing with each archive.php run. Last week it went up to 500M (piwik_archive_blob_2011_02 is 20M in size). I deleted all new rows that were added by error by the archive.php script, but today one week later, it is at 170M again.

What do you recommend? What should I do?

Thanks for the report. I added a comment in the related ticket Huge DB Disk space savings! Database purge should trigger on all archive_* tables at least once a day · Issue #3003 · matomo-org/matomo · GitHub

I confirm that there is a problem with the yearly reports. The following query shows this:

SELECT count(*), period FROM `piwik_archive_blob_2011_01` group by period
count(*) | period
39580	| 1
14277	| 2
10385	| 3
410471	| 4

But I think purging is just one half of the problem.
The other part is this:
We are now 19 june 2012. Why is the auto archiving process recalculating the statistics of 2011 at each run?
It should not. At least, if it is recalculating for those sites where the mem-limit is reached, the previous incomplete calculations should be removed.

If you need any other details, let me know.

I’m having some dramas since upgrading to 1.8.3.

2 tables are MUCH bigger than the others (see attached) and they seem to be regularly failing (I have to run REPAIR TABLE commands to fix them)

I’m assuming that this new scheduled task isn’t working as expected? It’s running inside a VPS with a fairly normal WHM/cPanel setup.

Any help is much appreciated.

Do you run the archive.php cron ? How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo

what is your archiving timeout, can you try 3600 seconds ?