How to tell if Piwik is using LOAD DATA INFILE?


#1

Hi there,

I’m trying to speed up report generation and I understand that if Piwik detects that LOAD DATA INFILE is working it can use that?

I was wondering how you can check if this is actually functional though, and functioning when using the application?

Edit: I have confirmed that the function works as expected, manually, using this website


#2

My understanding of LOAD DATA INFILE is for importing only your Apache logs.

So I’m not sure “report generation” would benefit from this MySQL function at all.

Maybe I didn’t understand what you’re trying to achieve.

Dali


#3

I may have misunderstood the function of the option in Piwik then. I read the following as referring to processing the reports. Is this not so?:


#4

And later on page How do I get LOAD DATA INFILE to work on my server? - Analytics Platform - Matomo it says “Piwik tries to import a lot of data in the database at once using a Mysql feature called “LOAD DATA INFILE””.

That’s why my feeling is that LOAD DATA INFILE is not relevant to Reports generation time.

But I’m no Piwik specialist…

I presume you already have a PHP accelerator (I use Xcache), that makes a huge difference.

Dali


(Matthieu Aubry) #5

Go to Settings > System check. If the line LOAD DATA INFILE is green then Piwik uses LOAD DATA INFILE.


#6

[quote=dali]I presume you already have a PHP accelerator (I use Xcache), that makes a huge difference.

Dali[/quote]

I use APC but yes PHP Accelerators make a huge amount of difference. We recently load tested our infrastructure and part of that included Piwik. I hadn’t set up the infrastructure so when the guy who was working on it was reporting poor Piwik performance, I asked if he’d set up PHP caching which he hadn’t. After getting that up and running everything was much better and we didn’t have to throw endless hardware at the servers.

[quote=matt]
Go to Settings > System check. If the line LOAD DATA INFILE is green then Piwik uses LOAD DATA INFILE.[/quote]

Yup it’s showing up as green on the Piwik server I’m working on
We have another Piwik infrastructure set up for a different product which we can’t get to show this. I’m not sure if this is because the DB and APP are on separate servers, or if it’s an issue with the particular version of PHP on that server, Ubuntu 12.04 LTS, because INFILE has been manually tested as working in the folder on the DB server.


(Matthieu Aubry) #7

Check the LOAD DATA INFILE FAQ

these instructions work for ubuntu 12.04


#8

Hi Matt,

Yup we’ve been through that and the only thing outstanding is upgrading PHP to a later version. If you say that it should work on Ubuntu 12.04LTS then I think it may be because we’re testing from an application server which is remotely connected to the DB on another server. We’re going to test locally on the DB server later.