Seperating the frontend javascript from backend database and analysis engine

Plugins & Platform
1

Hi,
I have an used case of doing analytic on a website which is in an isolated network. The site does not have connectivity to my piwik box. Is it possible to do some offline data collection on the site in a local database and then uploading the data to my piwik box for analysis ?
Thanks
Amit

2

Amit,

The only reasonable option that comes to mind is the ability to load in the server logs as mentioned here:

Any approach using the standard tracking system would require some “server-side” code on the local network. Setting aside some major technical challenges, you either need to send the JavaScript calls somewhere to be cached or you need to run the PHP tracking code to store hits. If you’re running some amount of code on the local network, you might as well run a full Piwik instance (occasionally backing up, transporting, and restoring the Piwik database if you were motivated to access the data outside the network).

3

Thanks for the reply. I am already trying out the logs upload. But ideally i would like to get all the javascript events too. I have multiple such clients in isolated networks so it wouldnt be very feasible to have a piwik installation per setup. I would like to collect the javascript events locally on each client’s database and then transfer these to a central piwik server.

Can you point me to the database code or document on how the javascript events are stored in the mysql database.

Thanks
Amit

4

The Javascript events aren’t like directly stored in the database. The Javascript on the client sends a request to Piwik’s Tracking API (see Tracking API - Analytics Platform - Matomo). These calls are processed by Piwik, data is added, and the result is stored in the database.

The simplest example of the problems you’d face is the timestamp. Piwik uses the server time of Javascript request as the time of an action/pageview. If you happened to store the Javascript calls somehow (say it were possible to cache them using an internet proxy), they wouldn’t reach the server until you forwarded them on. As a result, they would end up wit the wrong timestamp (and this would severely mess up some other Piwik logic).

There are ways around this particular issue (if you’re willing to send an admin token to a user’s browser), but this is just the simplest example of the problems you might face.

5

In summary… if you found a way to use a proxy to store and later send all the http requests, you might be able to get it working. In the default configuration, this would create some issues on the Piwik server, but most (and possibly all) can be solved if you’re willing to send the Admin token to the client’s computer (and you might be willing to do so if they’re closed networks).

However, the whole HTTP proxy possibility is way outside the scope of Piwik so it would be sheer luck to find someone here who knows how to make that work.

6

Good news, what you want to do is actually possible and do-able the “proper” way.

I just wrote this FAQ to assist you & others with the same needs: How do I configure Piwik Tracking for high reliability? - Analytics Platform - Matomo

let me know if that works!