Data migration by date


(Kelvin Latner) #1

Background:

  • We have Piwik/2.8.0 running on a dedicated Linux server in a distribution center, as our current Production environment.

  • We have Matomo/3.7.0 running on a private AWS/Cloud/Windows server, as our current test/pre-production/near-future Production environment.

  • Once we get all the user’s Piwik/2.8.0 data into the Matomo/3.7.0 database, we plan on using a DNS/c-name to redirect the Piwik/2.8.0 URL to the Matomo/3.7.0 URL. Our tests have shown the users are not only redirected to the Matomo/3.7.0 URL, but the data analytics that was going to the Piwik/2.8.0 environment get redirected to the Matomo/3.7.0 environment as well, eliminating the need for us to change the “code snippet” in 400+ site webpages…

  • If the migration/upgrade is successful, we’ll decommission the Piwik/2.8.0 environment; however, if need be, as a backout plan, we’ll switch the users back to the original Piwik/2.8.0 environment.

  • As a dry run pre-go-live test, when we’ve exported the Piwik/2.8.0 MySQL data (60G), and imported it into the Matomo/3.7.0 environment, it took almost a full day for the database to finish upgrading…

Point/Question:

  • Is there a way we can do the above process in advance, while users are still using the Piwik/2.8.0 environment (on a weekend when it’s slow), and then a week/weekend or so later, re-export only the new records as of the last export date, and re-import only the new records as of the last import date, thus reducing the migration/upgrade time for our users?

:smiley: