Importing logs before piwik creation date


#1

hello, i have recently created a piwik install, (feb 2014.) after the install was completed, i used the bulk tracking api to push older logs (may 2013) into the database. i can see the visitor logs but when i try to look at aggregated data, i get nothing.

in order to view the visitor logs, i manually changed the date range (not able to use the date selector widgets) but when using this same method to view the actual dashboard representation of the combined logs, i got an api error with no data for selected range

in order to fix this, i manually changed the time stamp in the piwik_site table, and while i am now able to use the date widgets to select times going back to the manually set time,and i no longer get an api error, i get zeroes for everything on the dash.

is this a supported use case? i can see in the mysql tables that something happened for may 2013:

mysql> select * from piwik_archive_numeric_2013_05;
±----------±---------------------------------------------------±-------±-----------±-----------±-------±--------------------±------+
| idarchive | name | idsite | date1 | date2 | period | ts_archived | value |
±----------±---------------------------------------------------±-------±-----------±-----------±-------±--------------------±------+
| 1 | done.VisitsSummary | 1 | 2013-05-01 | 2013-05-01 | 1 | 2014-02-12 15:00:13 | 1 |
| 1 | nb_uniq_visitors | 1 | 2013-05-01 | 2013-05-01 | 1 | 2014-02-12 15:00:13 | 7 |
| 1 | nb_visits | 1 | 2013-05-01 | 2013-05-01 | 1 | 2014-02-12 15:00:13 | 7 |
| 1 | nb_actions | 1 | 2013-05-01 | 2013-05-01 | 1 | 2014-02-12 15:00:13 | 27 |
| 1 | max_actions | 1 | 2013-05-01 | 2013-05-01 | 1 | 2014-02-12 15:00:13 | 13 |
| 1 | sum_visit_length | 1 | 2013-05-01 | 2013-05-01 | 1 | 2014-02-12 15:00:13 | 1402 |
| 1 | bounce_count | 1 | 2013-05-01 | 2013-05-01 | 1 | 2014-02-12 15:00:13 | 3 |
| 2 | done.VisitsSummary | 1 | 2013-05-02 | 2013-05-02 | 1 | 2014-02-12 15:00:13 | 1 |
| 3 | done.VisitsSummary | 1 | 2013-05-03 | 2013-05-03 | 1 | 2014-02-12 15:00:13 | 1 |
| 4 | done.VisitsSummary | 1 | 2013-05-04 | 2013-05-04 | 1 | 2014-02-12 15:00:13 | 1 |
| 4 | nb_uniq_visitors | 1 | 2013-05-04 | 2013-05-04 | 1 | 2014-02-12 15:00:13 | 1 |
| 4 | nb_visits | 1 | 2013-05-04 | 2013-05-04 | 1 | 2014-02-12 15:00:13 | 1 |
| 4 | nb_actions | 1 | 2013-05-04 | 2013-05-04 | 1 | 2014-02-12 15:00:13 | 1 |
| 4 | max_actions | 1 | 2013-05-04 | 2013-05-04 | 1 | 2014-02-12 15:00:13 | 1 |
| 4 | bounce_count | 1 | 2013-05-04 | 2013-05-04 | 1 | 2014-02-12 15:00:13 | 1 |
| 5 | done.VisitsSummary | 1 | 2013-05-05 | 2013-05-05 | 1 | 2014-02-12 15:00:13 | 1 |
| 5 | nb_uniq_visitors | 1 | 2013-05-05 | 2013-05-05 | 1 | 2014-02-12 15:00:13 | 3 |
| 5 | nb_visits | 1 | 2013-05-05 | 2013-05-05 | 1 | 2014-02-12 15:00:13 | 3 |
| 5 | nb_actions | 1 | 2013-05-05 | 2013-05-05 | 1 | 2014-02-12 15:00:13 | 11 |
| 5 | max_actions | 1 | 2013-05-05 | 2013-05-05 | 1 | 2014-02-12 15:00:13 | 7 |
| 5 | sum_visit_length | 1 | 2013-05-05 | 2013-05-05 | 1 | 2014-02-12 15:00:13 | 404 |
| 6 | done.VisitsSummary | 1 | 2013-05-06 | 2013-05-06 | 1 | 2014-02-12 15:00:13 | 1 |
| 7 | done.VisitsSummary | 1 | 2013-05-07 | 2013-05-07 | 1 | 2014-02-12 15:00:13 | 1 |
| 8 | done.VisitsSummary | 1 | 2013-05-08 | 2013-05-08 | 1 | 2014-02-12 15:00:13 | 1 |
| 9 | done.VisitsSummary | 1 | 2013-05-09 | 2013-05-09 | 1 | 2014-02-12 15:00:14 | 1 |
| 10 | done.VisitsSummary | 1 | 2013-05-10 | 2013-05-10 | 1 | 2014-02-12 15:00:14 | 1 |
| 11 | done.VisitsSummary | 1 | 2013-05-11 | 2013-05-11 | 1 | 2014-02-12 15:00:14 | 1 |
| 12 | done.VisitsSummary | 1 | 2013-05-12 | 2013-05-12 | 1 | 2014-02-12 15:00:14 | 1 |
| 13 | done.VisitsSummary | 1 | 2013-05-13 | 2013-05-13 | 1 | 2014-02-12 15:00:14 | 1 |
| 14 | done.VisitsSummary | 1 | 2013-05-14 | 2013-05-14 | 1 | 2014-02-12 15:00:14 | 1 |
| 15 | done.VisitsSummary | 1 | 2013-05-15 | 2013-05-15 | 1 | 2014-02-12 15:00:14 | 1 |
| 16 | done.VisitsSummary | 1 | 2013-05-01 | 2013-05-15 | 5 | 2014-02-12 15:00:14 | 1 |
| 17 | done.Actions | 1 | 2013-05-01 | 2013-05-15 | 5 | 2014-02-12 15:00:14 | 1 |
| 18 | done.Referrers | 1 | 2013-05-01 | 2013-05-15 | 5 | 2014-02-12 15:00:21 | 1 |
| 19 | done.VisitorInterest | 1 | 2013-05-01 | 2013-05-15 | 5 | 2014-02-12 15:00:22 | 1 |
| 20 | done.VisitTime | 1 | 2013-05-01 | 2013-05-15 | 5 | 2014-02-12 15:00:22 | 1 |
| 21 | done.UserSettings | 1 | 2013-05-01 | 2013-05-15 | 5 | 2014-02-12 15:00:23 | 1 |
| 22 | done.VisitsSummary | 1 | 2013-05-01 | 2013-05-05 | 5 | 2014-02-12 15:03:56 | 1 |
| 23 | done.Referrers | 1 | 2013-05-01 | 2013-05-05 | 5 | 2014-02-12 15:03:56 | 1 |
| 24 | done.UserSettings | 1 | 2013-05-01 | 2013-05-05 | 5 | 2014-02-12 15:03:56 | 1 |
| 25 | done.VisitTime | 1 | 2013-05-01 | 2013-05-05 | 5 | 2014-02-12 15:03:56 | 1 |
| 26 | done.VisitorInterest | 1 | 2013-05-01 | 2013-05-05 | 5 | 2014-02-12 15:03:56 | 1 |
±----------±---------------------------------------------------±-------±-----------±-----------±-------±--------------------±------+
42 rows in set (0.00 sec)


#2

think i fixed this:

setting the timestamp in the piwik_site table, and importing old logs worked and was not my problem, i believe it was the archiver having lack of memory. currently setting up cron task to get the log output


(Matthieu Aubry) #3

By design your use case should have worked, as Piwik when importing date in the past will automatically fix the website creation date to be the earliest date of the tracking data which was imported. let us know if you still have a question!