We are using 3 analytics tracking systems on a set of 100 websites. Our internal tracking system and Google Analytics agrees pretty closely on sessions. Matomo is missing about 4,800 sessions out of 20,700 (23%). Of the missing sessions 98% are Safari browsers across devices. About 43% have the same UA:
Perhaps stranger, not all users with Safari or even this User Agent are missed.
Safari users account for a sizable number of our orders and so simply ignoring their webtraffic is not an acceptable solution.
We are on the latest version of Matomo (just upgraded two weeks ago and the problem persists)
We have renamed the piwik.js file and server name
Do Not Track is off
We send a unique ID to all three systems so we can tell the individuals’ activity including that they hit a variety of web pages. There doesn’t appear to be a setup issue with the Matomo tracking script. I can see it firing in GTM
There are no javascript errors on the pages for these people.
Matomo is our system of record and we really need 100% of traffic tracked.
Hi Ken, one way to try and reproduce the issue would be to find the offending apache logs for your matomo requests (ie the requests to matomo.php or piwik.php) and then replay them using the log importer. Is this something you can do?
We reviewed the apache logs, and these sessions are not recorded at all.
We have a unique ID that we assign to each session and we searched the logs for these missing ID’s and found no record of the sessions associated at all.
(Also, we manage about 100 websites. we have found one website that is missing 50% of sessions. All missing sessions tie to Safari as the browser)
Interesting, if the requests aren’t even making it to the matomo server, then safari or the tracking code must be at fault. Either the requests are being sent to the wrong server (doubtful since this would affect every user agent, not just safari), or the tracking code isn’t working properly on safari. Would it be possible for you to post a link to the website in question?
We have a custom proprietary script that assigns each session an ID before the page begins to load
Tag Manager Fires
Tag manager loads both the Google Analytics script and the Matomo script at the same time. Both are asynchronous
We push the session ID into Matomo’s Custom Variables as part of the tag firing
We push the session ID into Google Analytics as a custom dimension before firing the GA tag
The Safari sessions show up in Google Analytics, but not in Matomo.
When we look at a session in Google Analytics that hit a rare page, that entire session is not in Matomo, nor does it appear in the server logs.
Again, we have DNT off, we have renamed the piwik.js, we are running the latest version.
This may be happening with other clients of yours, but they wouldn’t see the missing sessions without our unique setup of running two analytics systems fired on the same trigger at the same time, with a third system providing the session ID record.
If the logs don’t show the problem, then they won’t help find the problem. I can try to look at the website and see if I can reproduce the issue w/ the problematic safari version.
Are the requests to matomo/GA made server side? I see the code that sets the fingerprint, but I don’t see anything making HTTP requests that look like matomo tracking requests. I only see some requests to a /track endpoint. Does this send requests off to GA/matomo?
The logs aren’t going to be helpful if the requests never make it to matomo.
When looking at the website I don’t see anything making tracking requests directly to a matomo instance. Those requests would look something like https://mymatomo.com/piwik.php?idsite=...&url=...&action_name=...&.... If your web server logs for matomo (ie, the ones containing matomo.php/piwik.php requests) don’t show the data your missing, then it means the tracking requests for this problematic user agent are not actually getting to matomo (hence why nothing gets recorded).
My guess, since this affects only a certain safari browser, is that some browser setting is getting in the way of it sending the proper requests through your system. Unfortunately, I don’t know enough about your system to say exactly what could be wrong or where in your system the problem might manifest. Since it’s a browser related issue though, it may, as you said, affect other users, so we’d be interested in finding the root cause.
I can send you the logs. Our engineering team said that after they pulled the data, some of these visits are showing in the logs (originally they told me they were not). I’m going to work with them to see if patterns developed.
But I can send the logs to you to review as well. How do I send logs to you?
Hi, we have noticed the same issue and it happens with version 12_1_4. We can see the logs reaching the apache, but it does not appear in the “visits log”. Below is the User-Agent that comes for this when accessed using the “Captive Network Assistant” page. However, it is working fine when used using Safari or Chrome browser.
Captive Network Assistant User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16D57
Safari User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1"
Below is the User agent for Captive Network assistant for 12.3, it is being logged properly
“Mozilla/5.0 (iPhone; CPU iPhone OS 12_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148”
What I meant is that the Captive Portal browser in the past worked differently than the real safari on the same device, so I wouldn’t be surprised if there was a missing feature or bug in it that breaks your Matomo integration.