How to archive the reports for plugin?


(Jirapong) #1

When there is the call from API or Dashboard, I got this from the apache error_log file.

[2016-02-04 09:15:26] piwik.DEBUG: PluginsArchiver::callAggregateAllPlugins: Not archiving reports for plugin 'Resolution'. ["callAggregateAllPlugins","Resolution"] {"class":"Referrers","request_id":"25333"}
[2016-02-04 09:15:26] piwik.DEBUG: PluginsArchiver::callAggregateAllPlugins: Not archiving reports for plugin 'DevicePlugins'. ["callAggregateAllPlugins","DevicePlugins"] {"class":"Referrers","request_id":"25333"}
[2016-02-04 09:15:26] piwik.DEBUG: day archive, idSite = 1 (definitive archive), segment 'deviceType==tablet', report = 'Referrers', UTC datetime [2016-01-18 05:00:00 -> 2016-01-19 04:59:59] ["day",1,"definitive archive","deviceType==tablet","Referrers","2016-01-18 05:00:00","2016-01-19 04:59:59"] {"class":"Referrers","request_id":"25333"}
[2016-02-04 09:15:26] piwik.DEBUG: Db::query() executing SQL: DELETE FROM piwik_archive_numeric_2016_01 WHERE idarchive = ? AND (name = 'done5f13d0ad1705fbc536687f2f24768edf.Referrers') ["query","DELETE FROM piwik_archive_numeric_2016_01 WHERE idarchive = ? AND (name = 'done5f13d0ad1705fbc536687f2f24768edf.Referrers')"] {"class":"Referrers","request_id":"25333"}
[2016-02-04 09:15:26] piwik.DEBUG: Db::query() executing SQL: INSERT IGNORE INTO piwik_archive_numeric_2016_01 (idarchive, idsite, date1, date2, period, ts_archived, name, value)                   VALUES (?,?,?,?,?,?,?,?) ["query","INSERT IGNORE INTO piwik_archive_numeric_2016_01 (idarchive, idsite, date1, date2, period, ts_archived, name, value)\n                  VALUES (?,?,?,?,?,?,?,?)"] {"class":"Referrers","request_id":"25333"}
[2016-02-04 09:15:26] piwik.DEBUG: Db::fetchAll() executing SQL: SELECT idarchive, value, name, date1 as startDate FROM piwik_archive_numeric_2016_01                      WHERE idsite = ?                          AND date1 = ?                          AND date2 = ?                          AND period = ?                          AND ( (((name IN ('done5f13d0ad1705fbc536687f2f24768edf','done5f13d0ad1705fbc536687f2f24768edf.VisitsSummary','done5f13d0ad1705fbc536687f2f24768edf.Referrers')) AND (value IN (1,3,4))))                                OR name = 'nb_visits'                                OR name = 'nb_visits_converted')                           AND ts_archived >= ?                       ORDER BY idarchive DESC ["fetchAll","SELECT idarchive, value, name, date1 as startDate FROM piwik_archive_numeric_2016_01\n                     WHERE idsite = ?\n                         AND date1 = ?\n                         AND date2 = ?\n                         AND period = ?\n                         AND ( (((name IN ('done5f13d0ad1705fbc536687f2f24768edf','done5f13d0ad1705fbc536687f2f24768edf.VisitsSummary','done5f13d0ad1705fbc536687f2f24768edf.Referrers')) AND (value IN (1,3,4))))\n                               OR name = 'nb_visits'\n                               OR name = 'nb_visits_converted')\n                          AND ts_archived >= ? \n                     ORDER BY idarchive DESC"] {"class":"Referrers","request_id":"25333"}
[2016-02-04 09:15:26] piwik.DEBUG: Db::query() executing SQL: INSERT IGNORE INTO piwik_archive_numeric_2016_01 (idarchive, idsite, date1, date2, period, ts_archived, name, value)                   VALUES (?,?,?,?,?,?,?,?) ["query","INSERT IGNORE INTO piwik_archive_numeric_2016_01 (idarchive, idsite, date1, date2, period, ts_archived, name, value)\n                  VALUES (?,?,?,?,?,?,?,?)"] {"class":"Referrers","request_id":"25333"}
[2016-02-04 09:15:26] piwik.DEBUG: Db::query() executing SQL: INSERT IGNORE INTO piwik_archive_blob_2016_01 (idarchive, idsite, date1, date2, period, ts_archived, name, value)                   VALUES (?,?,?,?,?,?,?,?) ["query","INSERT IGNORE INTO piwik_archive_blob_2016_01 (idarchive, idsite, date1, date2, period, ts_archived, name, value)\n                  VALUES (?,?,?,?,?,?,?,?)"] {"class":"VisitTime","request_id":"0b938"}
[2016-02-04 09:15:26] piwik.DEBUG: PluginsArchiver::callAggregateAllPlugins: Initializing archiving process for all plugins [visits = 5784, visits converted = 0] ["callAggregateAllPlugins",5784,0] {"class":"Referrers","request_id":"25333"}

So, the Plugin report start to archive after that. Is there a way to include this to the console:archive?


(Kuba Bomba) #2

@Jirapong could you elaborate on that? Do you want to have some more verbose logging? Or maybe archive reports for a single plugin?


(Jirapong) #3

@quba - Thank you. I’m expecting a request from API should not run “Initializing archiving process for all plugins” but only read from what already archived in the piwik_archive_* table. So

  1. Is it possible to turn-off this in-time archiving?
  2. How can we make the auto archiving (crontab) to capture the data that avoid this in-time archiving

Thanks again. btw, May be because I’m very new to the platform and don’t fully understand how the API and archiving work.


(Kuba Bomba) #4

Please see this one: How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo


(Jirapong) #5

Thank you @quba. This already been set as well. I guess that the in-time archiving start when there is a new segment that not pre-defined in the segments[]= section. For example if API get is

https://xx.xxx.xxx/piwik/?callback=xxxxx_60&token_auth=xxxx&module=API&format=JSON&method=VisitsSummary.get&period=day&date=2016-01-10%2C2016-02-10&idSite=1&segment=deviceType%3D%3Dconsole

then

[2016-02-10 08:55:05] piwik.DEBUG: day archive, idSite = 1 (definitive archive), segment 'deviceType==console', report = 'VisitsSummary', UTC datetime [2016-01-18 05:00:00 -> 2016-01-19 04:59:59] ["day",1,"definitive archive","deviceType==console","VisitsSummary","2016-01-18 05:00:00","2016-01-19 04:59:59"] {"class":"VisitsSummary","request_id":"a4896"}
[2016-02-10 08:55:05] piwik.DEBUG: PluginsArchiver::callAggregateAllPlugins: Initializing archiving process for all plugins [visits = 1, visits converted = 0] ["callAggregateAllPlugins","1","0"] {"class":"VisitsSummary","request_id":"a4896"}

So, your configuration does not help.


(Kuba Bomba) #6

Segments are set to be archived in realtime by default. You can set these values in the config file:
browser_archiving_disabled_enforce = 1 enable_create_realtime_segments = 0

This assures that all segments will be pre-archived and browser archiving will be disabled. Would be also nice to set all segments to be pre-archived (either in the UI or directly in the DB (auto_archive column in the segment table).


(Jirapong) #7

Thank you very much!!