Rotate access log after log import

Support & Bugs
1

Hi,

I am using the access log import method for piwik analytics (rather than JS tracking) and I would like it to rotate the access log file (truncate it to 0 size, gzip the original and store it at some specified path) once the import has finished.

Is there a way to do it as part of the import process (some setting)? I could of course use a logrotate script, but I think there is going to be a time lag between the end of import process and the log file rotation, so some records (which will have been stored in the meantime) will be removed without having been imported.

I know AWStats has this option and has been rather useful (to us, at least).

Can you please advise me on it? I am on version 1.12.

Thanks,
Nick

2

It’s not possible (yet) to rotate logs as part of the script but we may add this later (if you would like us to implement this get in touch)

3

Thanks for the reply.

I am still very new to piwik and I am really getting acquainted to it.

For now I am going to try (this is a file, say web_accesslog, in /etc/logrotate.d/; I am in CentOS 6.4):


/path/to/website/access_log
{
    copytruncate
    missingok
    notifempty
    compress
    daily
    rotate 10
    prerotate
        /path/to/piwik_root/misc/cron/log_analysis_piwik
    endscript
}

where: /path/to/piwik_root/misc/cron/log_analysis_piwik:


python /path/to/piwik_root/www/misc/log-analytics/import_logs.py --url=http://site.example.com/piwik \
/path/to/website/access_log --idsite=1 --recorders=4 --enable-http-errors \
--enable-http-redirects --enable-static --enable-bots --enable-reverse-dns

/usr/bin/php /path/to/piwik_root/misc/cron/archive.php --url=http://site.example.com/piwik >> /path/to/piwik_root/archive-log/piwik-archive.log

If you have any suggestions regarding the above, I’ll appreciate it!

All the best!
Nick

4

Hello,

Note: I am now running Piwik 2.0.3 on CentOS 6.5 (on a professionally hosted virtual machine, hosting a website under NGINX / mysql / php-fpm).

The above approach (i.e. logrorate with prerotate script) seems to be working well, except that in some cases piwik does not seem to complete the analysis. Here is a monthly chart:

You can see that on Feb 6 and Feb 9, for some reason, something went wrong and stats were not produced. I have been unable to relate these issues to any specific events. Indeed, reports for these dates are empty.

Question 1: Could you suggest some reason why this may have happened?

Additionally, I am puzzled by the analysis produced for Jan 27 and Jan 29, where the Dashboard graph shows 0 unique visitors, while, in fact, there have been recorded (by piwik) unique visitors. For example, the “Visits by Server Time” table reveals that there were 15089 unique visitors on Jan 27.

Question 2: Why the unique visitors graph does not display the correct numbers for the afore mentioned dates?

Thanks and best regards,
Nick

5

Hmm,

Anyone?

To make things clearer, please let me rephrase Question 2: Can you please provide directions on how I can update the graph so as to display correctly the No. of unique visitors for the dates mentioned?

Thanks,
Nick