Hi, we’re just starting out with Piwik and have been practicing with an import log that spans exactly 12 hours (noon to midnight). When we import those logs into Piwik and view the “Visitors Times” page, in the Visits per server time section it shows 7 hours (12pm to 7pm) on the correct day (11/5 in this case) and 5 hours (12am to 5am) on 11/6. Our log file is showing -0500 for the time zone, our website in Piwik settings is set to UTC -5 and all time zones in PHP and Linux are set correctly. It’s almost like the importer hit midnight UTC (7pm EST) in the log and rolled to the next day even though there were still 5 hours of logs to go.
Update, just wiped everything out and rolled back to 1.12 (we were running 2.0 beta) and now when we import the data the times are correct. There’s a bug somewhere, not sure how to help troubleshoot it.
The issue is not in python parser, but somewhere on php side, since python sends correct date and time, according to UTC. But importer assumes that times are in server time, I suppose.
Piwik importer will accept times in the “Website timezones”. If you send times in UTC make sure the timezone is set to UTC for this website(s). If you can reproduce a bug we will be very happy to hear about (in 2.0-beta)
If you look into log parser code (misc/log_analytics/import_logs.py), you’ll see that it takes timezone stamps and subtracts it from the time. So, it converts time from server time to UTC. The bug is pretty well reproducible on any server with piwik where server timezone is different from UTC. Just import logs, and then go to piwik to see latest users actions and you’ll see that times for these events are off from the time specified in logs. And the difference is exactly what timezone stamp says. That is, if your server timezone is UTC+01, all imported times would be off for 1 hour.
But I think that it is good that parserd converts times to UTC, it is importer behaviour that should be changed. Because, I might import logs from different servers, or I had a situation when server timezone was changed when it was already running, so that in one file I had timestamps with different timezones. I think that when you export/import/transfer something, you should convert all platform-dependant values to platform independed ones.
Or at least, parser should have an option to set timezone to which it should convert times (converting it to UTC in the first place). Or at least an option ‘use local zimezone’ that would normalize all times to UTC and then add local server timezone offset.