SiteMigration plugin


#1

I tried this on two servers just to get permissions denied when I try using help command. Is there any documentation on this, please?

Thanks


#2

Got this part resolved. The original 644 would not allowed it. Once changed to 754, it worked. But… now, after running the command:

./console migration:site 1 -v

I get this:

Catchable fatal error: Argument 5 passed to Piwik\Plugins\SiteMigration\Commands\MigrateSite::ensureOptionsIsProvided() must be an instance of Piwik\Plugins\SiteMigration\Commands\callable, instance of Closure given, called in /usr/home/mysite/public_html/piwik/plugins/SiteMigration/Commands/MigrateSite.php on line 156 and defined in /usr/home/mysite/public_html/piwik/plugins/SiteMigration/Commands/MigrateSite.php on line 189

:-/

Help?!

Thanks

P.S.
Curiosity question: If I have two Piwik databases on the same server, and both of these have SiteID 1, when I run the migrate command, how does it know from which to which database I want to move the data? Thanks


(Matthieu Napoli) #3

Thank you for reporting this bug! This is an incompatibility with PHP 5.3, I have fixed it and released a new version of the plugin.

Regarding your question, let’s say you have:

  • Piwik 1
  • Piwik 2

If you run ./console migration:site 1 -v from Piwik 1 (i.e. from the directory of Piwik 1), then the data will be migrated from Piwik 1. The console command reads config/config.ini.php.

To know in which database you want to move the data, the console command will interactively ask you (so you have to give the database URL and login and password for the database of Piwik 2).


#4

Thanks. Now I get this (red highlight):

[Zend_Db_Statement_Mysqli_Exception]
Mysqli prepare error: Table ‘server_piwik.site’ doesn’t exist

Then, there’s this:

Exception trace:
() at /home/server/public_html/piwik/libs/Zend/Db/Statement/Mysqli.php:77
Zend_Db_Statement_Mysqli->_prepare() at /home/server/public_html/piwik/libs/Zend/Db/Statement.php:115
Zend_Db_Statement->__construct() at /home/server/public_html/piwik/libs/Zend/Db/Adapter/Mysqli.php:381
Zend_Db_Adapter_Mysqli->prepare() at /home/server/public_html/piwik/libs/Zend/Db/Adapter/Abstract.php:478
Zend_Db_Adapter_Abstract->query() at /home/server/public_html/piwik/libs/Zend/Db/Adapter/Abstract.php:575
Zend_Db_Adapter_Abstract->insert() at /home/server/public_html/piwik/plugins/SiteMigration/Helper/DBHelper.php:43
Piwik\Plugins\SiteMigration\Helper\DBHelper->executeInsert() at /home/server/public_html/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:56
Piwik\Plugins\SiteMigration\Migrator\TableMigrator->processRow() at /home/server/public_html/piwik/plugins/SiteMigration/Migrator/TableMigrator.php:47
Piwik\Plugins\SiteMigration\Migrator\TableMigrator->migrate() at /home/server/public_html/piwik/plugins/SiteMigration/Migrator/Migrator.php:144
Piwik\Plugins\SiteMigration\Migrator\Migrator->migrateSiteConfig() at /home/server/public_html/piwik/plugins/SiteMigration/Migrator/Migrator.php:107
Piwik\Plugins\SiteMigration\Migrator\Migrator->migrate() at /home/server/public_html/piwik/plugins/SiteMigration/Commands/MigrateSite.php:110
Piwik\Plugins\SiteMigration\Commands\MigrateSite->execute() at /home/server/public_html/piwik/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:252
Symfony\Component\Console\Command\Command->run() at /home/server/public_html/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:889
Symfony\Component\Console\Application->doRunCommand() at /home/server/public_html/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:193
Symfony\Component\Console\Application->doRun() at n/a:n/a
call_user_func() at /home/server/public_html/piwik/core/Console.php:60
Piwik\Console->Piwik{closure}() at /home/server/public_html/piwik/core/Access.php:454
Piwik\Access::doAsSuperUser() at /home/server/public_html/piwik/core/Console.php:61
Piwik\Console->doRun() at /home/server/public_html/piwik/vendor/symfony/console/Symfony/Component/Console/Application.php:124
Symfony\Component\Console\Application->run() at /home/server/public_html/piwik/console:30

Again, :-/, not sure where to look. The “old” Piwik is on the same server now, and it works, that is, I can log in. When it prompts me for a server, I just hit enter for a [localhost]. Then I enter the credentials.

Not sure why would it complain about server_piwik.site table. Why does it apply that “.site” to the end?

Thanks


(Matthieu Napoli) #5

“server_piwik” is the database. “site” is the table. Does that table exist?

If you want to see the options you can use for this command, run ./console migration:site


#6

Thanks.

Well, SITE table exists but with the prefix that has been setup when Piwik was installed. Where that “.site” comes from?

When I run the command:

./console migration:site 1 -v

… I know 1 stands for the SiteID that is about to be moved, but what word “site” stands for?

Thanks


(Matthieu Napoli) #7

Did you run ./console help migration:site ?

Because that will show you the option to use if the target database uses a table prefix.

Something like this: ./console migration:site 1 -v --db-prefix=“the_target_db_prefix_”

Regarding your second question, “migration:site” is the whole name of the command. The idea is that maybe later we can have other migration commands, like “migration:all-sites” or something.


#8

Thanks.

Once I used the option for targeted table prefix, it worked. But there’s only two days of data while it’s actually much more. When I login into the Piwik installation where the site was migrated to, graph shows flat zero line.


#9

Any chance to know why the data would not transfer, please?

Here is the log:

WARN SiteMigration[2014-12-03 22:42:11] [b2d90] Start transaction
WARN SiteMigration[2014-12-03 22:42:11] [b2d90] Migrating site config
WARN SiteMigration[2014-12-03 22:42:11] [b2d90] Loading existing actions
WARN SiteMigration[2014-12-03 22:42:12] [b2d90] Migrating log data - visits
WARN SiteMigration[2014-12-03 22:42:12] [b2d90] Migrating log data - link visit action
WARN SiteMigration[2014-12-03 22:42:13] [b2d90] Migrating log data - conversions and conversion items
WARN SiteMigration[2014-12-03 22:42:13] [b2d90] Migrating archive data
INFO SiteMigration[2014-12-03 22:42:13] [b2d90] Migrating archive 2014_11
INFO SiteMigration[2014-12-03 22:42:16] [b2d90] Migrating archive 2014_12
WARN SiteMigration[2014-12-03 22:42:16] [b2d90] Commit transaction

Is the above how it should look like?

Thanks


(Matthieu Napoli) #10

I don’t really know, could you have a look into the tables to compare them? See if the data is correctly migrated or not, and in which table it fails?

When I see the log you pasted I see that you have only 2 months of data (november and december) is that correct?


#11

When I see the log you pasted I see that you have only 2 months of data (november and december) is that correct?

No, the data there is from 2010 going forward. The archive tables are just those that were made when I tested the database to be sure that Piwik worked on the original installation. I deleted all archives to reduce the size.

Was the output on the screen right? It said WARN for all except the archive tables.

I just checked the log_visit table, and found just one raw with the new siteid 10 (this is what was asigned to the migrated site). This tells me that the data has not been migrated. How can this be fixed?
If tried multiple times, will the data be duplicated?

Thanks


(Matthieu Napoli) #12

If tried multiple times, will the data be duplicated?

It will be imported again in another new site, so it will be duplicated but not in the same site (except if you migrate from one existing site to another existing site but it doesn’t seem like it’s what you are doing).

I’m confused by the fact that you deleted some archive tables, archive tables are the processed data of Piwik. If you delete them then the migration can’t work I guess.


#13

except if you migrate from one existing site to another existing site

I have multiple Piwik installations. Tens of them. Now I’m consolidatiing some of them into a single Piwik installation. In this particular case, I have two existing installations: Piwik1 and Piwik2. Piwik1 is to be migrated to Piwik2. Piwik2 already had four sites configured in it.
After the (failed) migration attempt, now I have five sites in Piwik2, with the exception that the data for the fifth one is missing. That’s where I’m at right now.

In regards of the archive tables, they can be deleted at any time, as they’re recreated from the log_visit table which is the main data table about visits.

Why do you need archived tables for the migration preocess? How the migrations with this plugin works?

Also, what that WARN output meant, please?

Thanks


(Matthieu Napoli) #14

If you know how to recreate the archive tables then OK you don’t need all of them.

The WARN output is not really a warning, I understand how it can be confusing.

The migration is simply select all rows from a table and insert them in the new database (after changing the site id), you can see the code here: https://github.com/PiwikPRO/plugin-SiteMigration/tree/master/Migrator


#15

Thanks, and also thank you for making this tool. I’ll spend some time on checking on code to see if I can understand any so I get a better understanding.

In theory, if I’m not able to make the plugin work for me, I guess I could do it manually, based on how the plugin code works, right?

Thanks


(Matthieu Napoli) #16

Yes you could try manually.

Sorry not to be able to help more, I actually finished the plugin to be able to release it but it was already developed by other developers. I don’t know all the details on how/why it works that way :confused:


(Matthieu Aubry) #17

Hi @pwkuser if you have a problem with the plugin it would be super useful if you could provide a small database dump that would make reproducing the issue possible. Then if you create an issue in: Issues · PiwikPRO/plugin-SiteMigration · GitHub we will try to investigate it in coming weeks. Thanks


#18

It would be appreciated if those that know more could help here.

Thanks

P.S.
Ha, we posted at about the same time. I’ll see what I can do although it’s unlikely I’ll put my data there. Thanks anyway.


#19

Any chance to know how “idvisit” is handled when SiteMigration is ised?

Thanks