Massive Problems after Upgrade

Hi,

I upgraded our (low traffic!) piwik installation from 1.11 (!) to the current version 2.11.1. For the upgrade I moved my old piwik dir to piwik.old and created a fresh piwik directory, into which I copied the config.ini.php file from the old installation.
Then I started the database upgrade from the command line. This took some hours and stopped with a segmentation fault :frowning:
So I started the database upgrade again, it took some more hours, but this time it finished with a success message :slight_smile:
Now piwik is running again, but it is extremely slow (every page takes some minutes). So I tried to disable real-time reports and called core:archive manually / per cron. This takes some more hours, but for some sites it produces errors like this:

INFO CoreConsole[2015-02-27 14:03:15] Error: Got invalid response from API request: http:///piwik/index.php?module=API&method=API.get&idSite=4&period=week&date=last260&format=php&token_auth=d0c5bd34d9faedb915e0adf402c9e361&trigger=archivephp. The response was empty. This usually means a server error. This solution to this error is generally to increase the value of ‘memory_limit’ in your php.ini file. Please check your Web server Error Log file for more details.

I already increased the number of CPUs for the (virtual) server running piwik from 1 to 4 and as well the vRAM from 1GB to 4GB. I also increased the “memory_limit” to now 2048 MB, but the latter just means, it takes some more time until the error is produced.

I guess, our database is somehow broken. Can I check this for sanity? Perhaps I just need to remove some buggy entries, but which ones? :slight_smile:

Any ideas ?

Olaf.

P.S.: Our installation is really not big. I checked the database:
mysql> select idsite,count(idvisit) from p_log_visit group by idsite;
±-------±---------------+
| idsite | count(idvisit) |
±-------±---------------+
| 1 | 411233 |
| 2 | 17674 |
| 3 | 2119 |
| 4 | 5476 |
| 5 | 70 |
| 6 | 441 |
| 7 | 1367 |
| 8 | 6313 |
| 9 | 15429 |
| 10 | 1112 |
| 11 | 804 |
| 12 | 308 |
| 13 | 674 |
| 14 | 430 |
| 15 | 19602 |
| 16 | 688 |
| 17 | 5621 |
| 18 | 8469 |
| 19 | 36719 |
| 20 | 4710 |
| 21 | 473 |
| 23 | 4344 |
| 24 | 11876 |
| 25 | 2717 |
| 26 | 257 |
| 27 | 834 |
±-------±---------------+
26 rows in set (1 min 0.56 sec)

ARRRRGGHHH!!!
I realized, that the server was running for more than 3 years now without rotating the apache2 log! So this was more than 1.2 GB. After renaming it and restarting apache2, the server is much faster :slight_smile:
Now I restarted core:archive with --force-all-websites --force-all-periods and it went through in less than two hours and produced only two errors (for unimportant websites). Now I think I can clean up the database.

Olaf.

P.S.: I wonder why the old Piwik version had no problems with speed, if it turns out, that the issue was an apache2 misconfiguration…

Hi there,

Also in case you are using PHP 5.3 please upgrade to php 5.5 which brings many performance improvements

Hi,

after fixing the apache2 log problem, I still got errors like this one (from the archive cron job):

ERROR CoreConsole[2015-03-02 13:41:12] Got invalid response from API request: http:///piwik/index.php?module=API&method=API.getDefaultMetricTranslations&format=original&serialize=1&trigger=archivephp. The response was empty. This usually means a server error. This solution to this error is generally to increase the value of ‘memory_limit’ in your php.ini file. Please check your Web server Error Log file for more details.
ERROR CoreConsole[2015-03-02 13:41:12] The Piwik URL http:///piwik/index.php does not seem to be pointing to a Piwik server. Response was ‘’.
ERROR CoreConsole[2015-03-02 13:41:12] The Piwik URL http:///piwik/index.php does not seem to be pointing to a Piwik server. Response was ‘’.


But now I upgraded the PHP 5.3.17 to PHP 5.6.6. Since this update the piwik-archive runs in just 3 minutes for all periods and all 27 websites (took several hours before), and I don’t get any more errors like the above one.

Olaf.

P.S.: I think, you should mention, that Piwik is already incompatible with PHP 5.3 :-]