Invalidate Report with Custom Dimensions

General questions
#1

Hey @matthieu, any Matomo Team Member or anyone reading this post that can help.

Are we able to invalidate a report with a Custom Dimension as a segment? I am using Custom Dimensions a lot and need to invalidate so historical data shows for recently built pre-processed segments.

I have tried with an API call and receive a message stating Custom Dimensions are not supported, really hope I am wrong here.

I can invalidate other reports, just seeing this message when referring to Custom Dimensions.

Please help answer this, one way or the other…?

#2

Any luck on this? We have a custom dimension segment and we run this command to invalidate the data after importing data via the HTTP API.

/console core:invalidate-report-data --sites=2110 --dates=2018-06-01,2018-08-17 --segment=dimension15=^en-GB

However, the data does not get re-calculated after re-archiving. Now “All Visits” and this segment do not agree.

#3

Hi. Unfortunately neither the client nor server API request was successful for me. I am going to install the “invalidate reports” plugin as I have read others have had more success when using the plugin over the API.

I’ll report back when I have tested this, unless you beat me to it :wink:

#4

Hi, thanks for the reply. The plugin, nor the API work for us specifically with segments of custom dimensions. We will likely manually delete the data we need to invalidate. I’ll report back here if I can get a manual solution in place. It’s a priority for us because the client now sees two versions of their revenue totals.

#5

Oh dear, that’s not good! I understand the urgency.

Did the plugin work for you for existing core Dimensions - Page Url as an example? Could you invalidate without segmentation for the same period, perhaps you have tried that already with no success.

I also very much need to invalidate data via a custom dimension segment. I see a message returned from the http API stating Custom Dimensions aren’t supported and hoped the plugin would do the job.

#6

We have issues invalidating data regularly. Sometimes it works, sometimes it doesn’t. I’m not sure I see the utility in “invalidating” data anyway. If it’s invalid, just delete it! (That’s probably a naive statement, but…hey)

It is often resolved by using the plugin v.s. the command line. The “big stick” we have is deleting directly from the database tables by deleting data for a specific site, or by dropping the table entirely and re-archiving. That seems to always work when nothing else does. I had not tried that with this particular custom dimension segment issue until today.

We manually deleted the data from the archive tables for the date range we were interested in:

DELETE FROM `piwik_archive_blob_2018_06` WHERE idsite = 1234;
DELETE FROM `piwik_archive_numeric_2018_06` WHERE idsite = 1234;

By the way, in other situations we have also just dropped the entire table and re-archived all sites. Just be sure to capture both blob and numeric tables for the date range in question.

Then ran the archive on the command line

./console core:archive --force-idsites=1234

This, by the way, is why we never delete visitor logs. Disk space be damned.

#7

I hear you on the log storage for sure.

I have to rely on pre-processing quite a bit for performance issues and so invalidation is important to “look back” for trend.

I’ll take your points about direct deletion in the dB, May have to rely on that unless Matomo can ever incorporate their custom dimensions capability into this area.

Thanks for the info :+1:t2:

#8

Deleting the data from the dB and rearchiving worked as suggested.