I’m writing a plugin that logs bots detected by built-in DeviceDetector.
Problem is with non-day aggregation of my datatables. Database have many string columns, some of them are displayed in UI. First I was putting them all into Row::COLUMNS, but it caused more problems with archiver trying to sum them all by default (I am using RecordBuilder btw), so I left ‘label’ and ‘nb_visits’ in columns and moved everything else to Row::METADATA (which I convert dynamically to columns in priority filter inside report’s configureView() just to display in UI).
Now the problem is how to define metadata aggregation rules to choose which meta to keep.
RecordBuilder has $columnAggregationOps, it seems that it is used for both columns and metadata. Also a Record built in getRecordMetadata() has setBlobColumnAggregationOps(). Are they the same? Different? How to archive all this mess? The string data should be really taken from first or last record, or whatever, it’s all the same (unique for a bot label the whole dataset is grouped by).
If the string value is identical within each bot-label group, using a first/last aggregation should be sufficient. I’d keep those fields in metadata and define an explicit aggregation rule for them, rather than letting RecordBuilder try to sum them. Also, setBlobColumnAggregationOps() is worth checking separately from $columnAggregationOps, since they may target different record types.
There are ‘first’/‘last’ options? Not a single plugin in Matomo has these strings, so I doubt it. I added custom functions and they are working fine. I had to study sources and other plugins to understand how this actually works. $columnAggregationOps is a default for the Report, setBlobColumnAggregationOps() is an override for particular Record in a Report. These 2 and also datatable’s aggregation metadata COLUMN_AGGREGATION_OPS_METADATA_NAME can all be assigned the same array for aggregation of everything (rules apply to both columns and metadata).
My problem was disabled on-demain archives from browser requests as core:archive does not generate custom ranges. Also when selecting a period in Matomo UI and then getting starting and ending time using Period class I was getting 00:00:00 and 23:59:59 in UTC, but that’s not correct even though site’s timezone was used to build Period. UTC offset must be subtracted to get actual UTC times that are consistent with my db datetimes.
Since those string values are identical within each bot label group, using the first or last value should be sufficient. I’d keep them in metadata and define an explicit aggregation rule for those fields to avoid Matomo applying numeric aggregation by default.
The string data should be really taken from first or last record, or whatever, it’s all the same (unique for a bot label the whole dataset is grouped by).