Using the new Migration Plugin to consolidate historic data with current data for one site. Plugin installed and connected fine. I did get these errors, and the migration never completed
WARNING [2019-02-05 21:32:33] 624 /var/www/html/piwik/plugins/Migration/Migrations/LogMigration.php(127): Notice - Undefined index: last_idlink_va - Matomo 3.8.1 - Please report this message in the Matomo forums: https://forum.matomo.org (please do a search first as it might have been reported already)
[Zend_Db_Statement_Exception]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '2484-donefea44bece172bc9696ae57c26888bf8a.VisitsSummary' for key 'PRIMARY'
[PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '2484-donefea44bece172bc9696ae57c26888bf8a.VisitsSummary' for key 'PRIMARY'
Any thoughts on how to move past this? I know the WARNING may not be as serious as the second portion. running the --skip-archives returns just the warnings and “Error: error or warning logs detected, exit 1”
Running with --skip-logs returns the second error
I’ve released a new version that should fix the notice re last_idlink_va.
Re the archive constraint violation: Is there any chance you could paste what the SQL query select * from piwik_sequence or select * from matomo_sequence (depending on table prefix) returns on the target database?
1 Like
Sure, I started the process over again today after I noticed that the migration plugin created a new site for each time I ran the migration with error. Same errors, just a different string displayed
[Zend_Db_Statement_Exception]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '2972-done' for key 'PRIMARY'
[PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '2972-done' for key 'PRIMARY'
and the output of the select statement is:
MariaDB [matomo]> select * from piwik_sequence;
+-------------------------------+-------+
| name | value |
+-------------------------------+-------+
| piwik_archive_numeric_2005_01 | 1 |
| piwik_archive_numeric_2006_01 | 1 |
| piwik_archive_numeric_2007_01 | 1 |
| piwik_archive_numeric_2008_01 | 1 |
| piwik_archive_numeric_2008_12 | 2 |
| piwik_archive_numeric_2009_01 | 1 |
| piwik_archive_numeric_2010_01 | 1 |
| piwik_archive_numeric_2010_05 | 1 |
| piwik_archive_numeric_2010_09 | 1 |
| piwik_archive_numeric_2010_10 | 1 |
| piwik_archive_numeric_2010_11 | 1 |
| piwik_archive_numeric_2010_12 | 1 |
| piwik_archive_numeric_2011_01 | 14173 |
| piwik_archive_numeric_2011_02 | 1 |
| piwik_archive_numeric_2011_03 | 1 |
| piwik_archive_numeric_2011_04 | 1218 |
| piwik_archive_numeric_2011_05 | 5519 |
| piwik_archive_numeric_2011_06 | 7899 |
| piwik_archive_numeric_2011_07 | 9296 |
| piwik_archive_numeric_2011_08 | 9679 |
| piwik_archive_numeric_2011_09 | 8957 |
| piwik_archive_numeric_2011_10 | 9652 |
| piwik_archive_numeric_2011_11 | 8949 |
| piwik_archive_numeric_2011_12 | 9164 |
| piwik_archive_numeric_2012_01 | 29144 |
| piwik_archive_numeric_2012_02 | 7199 |
| piwik_archive_numeric_2012_03 | 5358 |
| piwik_archive_numeric_2012_04 | 7492 |
| piwik_archive_numeric_2012_05 | 8178 |
| piwik_archive_numeric_2012_06 | 8236 |
| piwik_archive_numeric_2012_07 | 9556 |
| piwik_archive_numeric_2012_08 | 7345 |
| piwik_archive_numeric_2012_09 | 8771 |
| piwik_archive_numeric_2012_10 | 9733 |
| piwik_archive_numeric_2012_11 | 10968 |
| piwik_archive_numeric_2012_12 | 13432 |
| piwik_archive_numeric_2013_01 | 16647 |
| piwik_archive_numeric_2013_02 | 11855 |
| piwik_archive_numeric_2013_03 | 11574 |
| piwik_archive_numeric_2013_04 | 13113 |
| piwik_archive_numeric_2013_05 | 12501 |
| piwik_archive_numeric_2013_06 | 12718 |
| piwik_archive_numeric_2013_07 | 11837 |
| piwik_archive_numeric_2013_08 | 12938 |
| piwik_archive_numeric_2013_09 | 13344 |
| piwik_archive_numeric_2013_10 | 13041 |
| piwik_archive_numeric_2013_11 | 13195 |
| piwik_archive_numeric_2013_12 | 13734 |
| piwik_archive_numeric_2014_01 | 32533 |
| piwik_archive_numeric_2014_02 | 13277 |
| piwik_archive_numeric_2014_03 | 15650 |
| piwik_archive_numeric_2014_04 | 16211 |
| piwik_archive_numeric_2014_05 | 16544 |
| piwik_archive_numeric_2014_06 | 17775 |
| piwik_archive_numeric_2014_07 | 16992 |
| piwik_archive_numeric_2014_08 | 17113 |
| piwik_archive_numeric_2014_09 | 17308 |
| piwik_archive_numeric_2014_10 | 17000 |
| piwik_archive_numeric_2014_11 | 7172 |
| piwik_archive_numeric_2014_12 | 1763 |
| piwik_archive_numeric_2015_01 | 1921 |
| piwik_archive_numeric_2015_02 | 1651 |
| piwik_archive_numeric_2015_03 | 1774 |
| piwik_archive_numeric_2015_04 | 1734 |
| piwik_archive_numeric_2015_05 | 4619 |
| piwik_archive_numeric_2015_06 | 6561 |
| piwik_archive_numeric_2015_07 | 6595 |
| piwik_archive_numeric_2015_08 | 7235 |
| piwik_archive_numeric_2015_09 | 6166 |
| piwik_archive_numeric_2015_10 | 6395 |
| piwik_archive_numeric_2015_11 | 6663 |
| piwik_archive_numeric_2015_12 | 5629 |
| piwik_archive_numeric_2016_01 | 7794 |
| piwik_archive_numeric_2016_02 | 6851 |
| piwik_archive_numeric_2016_03 | 7243 |
| piwik_archive_numeric_2016_04 | 6252 |
| piwik_archive_numeric_2016_05 | 12657 |
| piwik_archive_numeric_2016_06 | 11831 |
| piwik_archive_numeric_2016_07 | 13210 |
| piwik_archive_numeric_2016_08 | 15124 |
| piwik_archive_numeric_2016_09 | 14773 |
| piwik_archive_numeric_2016_10 | 16159 |
| piwik_archive_numeric_2016_11 | 14619 |
| piwik_archive_numeric_2016_12 | 13985 |
| piwik_archive_numeric_2017_01 | 17012 |
| piwik_archive_numeric_2017_02 | 11185 |
| piwik_archive_numeric_2017_03 | 11054 |
| piwik_archive_numeric_2017_04 | 10570 |
| piwik_archive_numeric_2017_05 | 11498 |
| piwik_archive_numeric_2017_06 | 11962 |
| piwik_archive_numeric_2017_07 | 11862 |
| piwik_archive_numeric_2017_08 | 12030 |
| piwik_archive_numeric_2017_09 | 10513 |
| piwik_archive_numeric_2017_10 | 10911 |
| piwik_archive_numeric_2017_11 | 3606 |
| piwik_archive_numeric_2017_12 | 2965 |
| piwik_archive_numeric_2018_01 | 3389 |
| piwik_archive_numeric_2018_02 | 2897 |
| piwik_archive_numeric_2018_03 | 3211 |
| piwik_archive_numeric_2018_04 | 3359 |
| piwik_archive_numeric_2018_05 | 3392 |
| piwik_archive_numeric_2018_06 | 3248 |
| piwik_archive_numeric_2018_07 | 3438 |
| piwik_archive_numeric_2018_08 | 4050 |
| piwik_archive_numeric_2018_09 | 3613 |
| piwik_archive_numeric_2018_10 | 4099 |
| piwik_archive_numeric_2018_11 | 3941 |
| piwik_archive_numeric_2018_12 | 5273 |
| piwik_archive_numeric_2019_01 | 81419 |
| piwik_archive_numeric_2019_02 | 19644 |
+-------------------------------+-------+
110 rows in set (0.05 sec)
Cheers, looks all good. I’ve tried several cases and can’t reproduce it. Not even an idea how this could happen. Maybe somehow some corrupt DB entry in the source database.
I’ve released 3.0.2 with more logging. Can you update and check at which DB table it stops? There should be outputs like
Starting to migrate archive table archive_blob_2018_11 at 2019-02-06 22:23:54
Can you then do eg a SHOW CREATE TABLE piwik_archive_blob_2018_11 depending on which table fails? It might not help much but at the same time I have no idea what could cause the error. You might also want to do a query from that table like select idsite from piwik_archive_blob_2018_11 where idarchive=2972 where you replace the number 2972 with the duplicate ID it mentions. Wondering if this is the same ID as the target idsite, or from another site.
Ok, the new logging is helpful and the warnings are gone. I believe I may have misunderstood the functionality of this migration plugin.
Prior to using the plugin, I manually dumped data from an old Piwik installation to a new one. We didn’t make the url switch until a week after the manual dump, so one week’s worth of data stayed on the old server and I was thinking that a migration would combine the data from the old installation into the new one. I don’t think this plugin will actually merge data for idsite=1 on Server/DB A into idsite=1 on Server/DB B as the migration automatically assigns a new idsite number upon migration.
It can’t combine the data unfortunately. Thanks for letting us know.
1 Like