Piwik Upgrade Without Missing Logs


#1

Hello,

I’m wondering if anyone has already done this, or tips how achieve the following.

Upgrade a Piwik installation without any data loss. According to the documentation - and common sense - there should be no data written to the tables while they are upgraded, this means deactivating logging one way or another.

Depending on the site this can take some time. During this time no data is logged, so your statistics are missing for this period. What are options to counteract this and don’t miss any data?

One way I can think of is to
[ul]
[li] first take snapshot of the current installation
[/li][li] perform the upgrade on the snapshot (original copy continues to run)
[/li][li] go live with the upgraded snapshot
[/li][li] perform upgrade on original copy
[/li][li] copy diff on piwik_log_* tables over (handling possible collisions, e.g. in piwik_log_action)
[/li][/ul]

You’d probably only get some unique visitor violations during the copy phase.

Does anyone have suggestions and / or has done this already? How do larger installations deal with this?

The whole thing for me is currently a theoretical scenario, because we can afford missing a few minutes of logs. But the website is growing and we are developing zero downtime upgrade strategies for the other components and ideally Piwik should be integrated into this.

ciao,
elm


(Matthieu Aubry) #2

I suggest that you wait for Bulk load Piwik logs with documented API: improved tracking performance, allow performance testing · Issue #5554 · matomo-org/matomo · GitHub to be implemented (potentially sponsoring some funds towards its development) as it will then log into a file (or a queue service) and then replay the logs later on. So, during upgrade, it will still log data in the file, then you could import stats after upgrade was finished, without any data loss