Slow performance

General questions
1

I have Piwik installed on a Linux ubuntu 16 ts.micro machine. Using MySql DB.

All working ok - I have imported 6 years worth of logs. We’re not talking huge logs - 5-6000 visits per month.

I can run reports covering a day’s visits but reports covering a month are slow to display and for certain widgets I get an error:

Oops… there was a problem during the request. Maybe the server had a temporary issue, or maybe you requested a report with too much data.

Anything longer than a month just whirs away with no response until I give up!

I took Piwik to be an analytics engine so I assumed that the import of logs was the ETL for the system and then some kind of analysis engine kicked in to create aggregates of the data. The reports then just run off the aggregates. Is this the case?

If so, then why is it so slow or not responding at all?

I want to produce reports like: monthly visits over a 5 year period.

Have I missed something? Does my machine need more power for that level of logs?

2

Hi,

Have you set up a cronjob for archiving (=generating reports) and then disabled browser triggering archiving?
See here for more information:
https://piwik.org/docs/setup-auto-archiving/#disable-browser-triggers-for-piwik-archiving-and-limit-piwik-reports-to-updating-every-hour

You can find more preformance tips here:
https://piwik.org/docs/optimize-how-to/

1 Like
3

Thanks Lukas,

So archiving creates the aggregates?

The documentation on archiving is not clear to me.

Let’s forget cron job for now. What would the command be to manually run an archive - something like this?:

sudo su www-data -s /bin/bash -c “/usr/bin/php5 /path/to/piwik/console core:archive --url=http://example.org/piwik/”

What is /path/to/piwik/console - how do I find what that is? Is it URL or file system path?

I assume some kind of feedback is produced as to success or failure?

4

Hi,

Exactly, by default Piwik generates all Reports live when you request them, but with large amounts of data that can be slow. So you can run the command every n minutes and therefore have nearly-live reports without delay on load.

That should work even though it seems a bit overly complicated for me. I’d run the following:
sudo -u www-data /usr/bin/php7.1 /var/www/piwik/console core:archive --url=https://piwik.example.com/
Keep in mind:

  • You need to run to use the same php version as the webserver is using
  • the console file should be in the root folder of piwik (the same as piwik.php and piwik.js)
  • the url should be the URL to your piwik instance.
5

I assume I only need a Cron job if i’m doing live website logging rather than batch job from log files.

I used this command in the end:

sudo su www-data -s /bin/bash -c “/usr/bin/php /var/www/html/piwik/console core:archive --url=https://piwik.mydomain.co.uk/piwik/”

I’m using php 7 and was getting memory_limit errors - I edited the php.ini files under /etc/php/7.0/cli and /etc/php/7.0/apache2 so that memory_limit=1024M and also upped my AWS instance memory to 2GB and all was then ok.

Reports now working really fast - many thanks!