we use matomo and track about 600.000 pageviews per month with our site. Ne to this we track a lot of events to matomo (about 100.000 events per month).
With these events we have often the problem of missing events in matomo. For example if we have 100 registration events in our internal event tracking system we just have 92 registration events in matomo.
We don’t know why this happens and how we can handle and solve this.
We don’t see any “tracking failures” in matomo.
When we look at the load factor of the server everything seems ok. But is it anyway possible that we send to many request to matomo so not all data/events can be stored in matomo?
Is there a way to see this?
And: Would you recommend to use the plugin for queued tracking in our case?
Can you check also the error log file of Matomo / Apache if you see any error?
Also, for the test you identified (92/100), try to check if all requests arrived to your server (in the access log file)
In my own experience (around 4,000,000 hits per day), I had to add the queued tracking because the Matomo UI was very slow AND the live tracking log had a high lateness (around 4 hours). We have 3 tracking servers, and the database is hosted on another server…
How many servers do you use for tracking?
Thanks for your reply. I will check the log-files and give some feedback about this.
Our matomo is a self hosted bitnami server (hosted in google cloud) with 2 vcore-cpu and 8 gb ram.
It seems that there was an update this week from 4.11.0 to 4.12.2 (I don’t know what triggered the update; is there an cronjob/automatism which auto-updates matomo? I think we don’t started the update manually) and since this week/update our system has often a cpu load with 90 % for around 20 minutes.
I will try to delete some segments (we have around 50 segments (36 pre-processed, 14 processed in real-time)) - maybe this helps.
We currently also get the following error in system check: Last Successful Archiving Completion
In log viewer we see errors like these:
ERROR CoreConsole 2022-10-27 10:24:20 UTC 13303 78 total errors during this script execution, please investigate and try and fix these errors.
INFO CoreConsole 2022-10-27 10:24:20 UTC 13303 Error: Error unserializing the following response from ?module=API&method=CoreAdminHome.archiveReports&idSite=6&period=year&date=2022-01-01&format=json&segment=eventCategory%3D%3DExperimentViewed%3BeventAction%3D%3Dhomepage_hero_button%3BeventName%3D%3Dv1&trigger=archivephp: 'PHP Fatal error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 852320064 bytes) in /opt/bitnami/apps/matomo/htdocs/libs/Zend/Db/Statement/Mysqli.php on line 250 {"result":"error","message":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 852320064 bytes) on \/opt\/bitnami\/apps\/matomo\/htdocs\/libs\/Zend\/Db\/Statement\/Mysqli.php(250) #0 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\\Plugins\\Actions\\Archiver->callAggregateDayReport() ","backtrace":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 852320064 bytes) on \/opt\/bitnami\/apps\/matomo\/htdocs\/libs\/Zend\/Db\/Statement\/Mysqli.php(250)\n#0 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\\Plugins\\Actions\\Archiver->callAggregateDayReport()\n\n#0 [internal function]: Piwik\\Plugins\\CorePluginsAdmin\\Controller->safemode(Array)\n#1 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(631): call_user_func_array(Array, Array)\n#2 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(169): Piwik\\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n#3 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(100): Piwik\\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n#4 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(275): Piwik\\FrontController::generateSafeModeOutputFromError(Array)\n#5 [internal function]: Piwik\\FrontController::triggerSafeModeWhenError()\n#6 {main}"}{"result":"error","message":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 852320064 bytes) on \/opt\/bitnami\/apps\/matomo\/htdocs\/libs\/Zend\/Db\/Statement\/Mysqli.php(250) #0 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\\Plugins\\Actions\\Archiver->callAggregateDayReport() ","backtrace":"Allowed memory size of 805306368 bytes exhausted (tried to allocate 852320064 bytes) on \/opt\/bitnami\/apps\/matomo\/htdocs\/libs\/Zend\/Db\/Statement\/Mysqli.php(250)\n#0 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/ArchiveProcessor\/PluginsArchiver.php(164): Piwik\\Plugins\\Actions\\Archiver->callAggregateDayReport()\n\n#0 [internal function]: Piwik\\Plugins\\CorePluginsAdmin\\Controller->safemode(Array)\n#1 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(631): call_user_func_array(Array, Array)\n#2 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(169): Piwik\\FrontController->doDispatch('CorePluginsAdmi...', 'safemode', Array)\n#3 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(100): Piwik\\FrontController->dispatch('CorePluginsAdmi...', 'safemode', Array)\n#4 \/opt\/bitnami\/apps\/matomo\/htdocs\/core\/FrontController.php(275): Piwik\\FrontController::generateSafeModeOutputFromError(Array)\n#5 [internal function]: Piwik\\FrontController::triggerSafeModeWhenError()\n#6 {main}"}'