Safari User Agent 98% visitor and session loss

We are using 3 analytics tracking systems on a set of 100 websites. Our internal tracking system and Google Analytics agrees pretty closely on sessions. Matomo is missing about 4,800 sessions out of 20,700 (23%). Of the missing sessions 98% are Safari browsers across devices. About 43% have the same UA:

Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1
https://developers.whatismybrowser.com/useragents/parse/1246394-safari-ios-iphone-webkit

Perhaps stranger, not all users with Safari or even this User Agent are missed.

Safari users account for a sizable number of our orders and so simply ignoring their webtraffic is not an acceptable solution.

We are on the latest version of Matomo (just upgraded two weeks ago and the problem persists)
We have renamed the piwik.js file and server name
Do Not Track is off
We send a unique ID to all three systems so we can tell the individuals’ activity including that they hit a variety of web pages. There doesn’t appear to be a setup issue with the Matomo tracking script. I can see it firing in GTM
There are no javascript errors on the pages for these people.

Matomo is our system of record and we really need 100% of traffic tracked.

Obtained a Safari 12_1_4 Ipad and tested it across 11 websites. All of them were recorded accurately in Matomo.

I am struggling to reproduce this issue

1 Like

Hi Ken, one way to try and reproduce the issue would be to find the offending apache logs for your matomo requests (ie the requests to matomo.php or piwik.php) and then replay them using the log importer. Is this something you can do?

1 Like

We reviewed the apache logs, and these sessions are not recorded at all.

We have a unique ID that we assign to each session and we searched the logs for these missing ID’s and found no record of the sessions associated at all.

(Also, we manage about 100 websites. we have found one website that is missing 50% of sessions. All missing sessions tie to Safari as the browser)

Interesting, if the requests aren’t even making it to the matomo server, then safari or the tracking code must be at fault. Either the requests are being sent to the wrong server (doubtful since this would affect every user agent, not just safari), or the tracking code isn’t working properly on safari. Would it be possible for you to post a link to the website in question?

ATTSavings.com is the example we are using.

Our setup is as follows:

  • We have a custom proprietary script that assigns each session an ID before the page begins to load
  • Tag Manager Fires
  • Tag manager loads both the Google Analytics script and the Matomo script at the same time. Both are asynchronous
  • We push the session ID into Matomo’s Custom Variables as part of the tag firing
  • We push the session ID into Google Analytics as a custom dimension before firing the GA tag

The Safari sessions show up in Google Analytics, but not in Matomo.

When we look at a session in Google Analytics that hit a rare page, that entire session is not in Matomo, nor does it appear in the server logs.

Again, we have DNT off, we have renamed the piwik.js, we are running the latest version.

This may be happening with other clients of yours, but they wouldn’t see the missing sessions without our unique setup of running two analytics systems fired on the same trigger at the same time, with a third system providing the session ID record.

We have the logs for the day (nginx, not apache)

How can I send them to you?

If the logs don’t show the problem, then they won’t help find the problem. I can try to look at the website and see if I can reproduce the issue w/ the problematic safari version.

Are the requests to matomo/GA made server side? I see the code that sets the fingerprint, but I don’t see anything making HTTP requests that look like matomo tracking requests. I only see some requests to a /track endpoint. Does this send requests off to GA/matomo?

How do I upload the logs? compressed file type .gz is not permitted.

We’re absolutely making direct Matomo tracking requests. I’m not sure how to answer your last question

The logs aren’t going to be helpful if the requests never make it to matomo.

When looking at the website I don’t see anything making tracking requests directly to a matomo instance. Those requests would look something like https://mymatomo.com/piwik.php?idsite=...&url=...&action_name=...&.... If your web server logs for matomo (ie, the ones containing matomo.php/piwik.php requests) don’t show the data your missing, then it means the tracking requests for this problematic user agent are not actually getting to matomo (hence why nothing gets recorded).

My guess, since this affects only a certain safari browser, is that some browser setting is getting in the way of it sending the proper requests through your system. Unfortunately, I don’t know enough about your system to say exactly what could be wrong or where in your system the problem might manifest. Since it’s a browser related issue though, it may, as you said, affect other users, so we’d be interested in finding the root cause.

Remember we have renamed matomo (to Deckard.clearlink.com) to avoid blocking mechanisms.

I can send you the logs. Our engineering team said that after they pulled the data, some of these visits are showing in the logs (originally they told me they were not). I’m going to work with them to see if patterns developed.

But I can send the logs to you to review as well. How do I send logs to you?

I sent you a message on how to get the logs to me.

Hi, we have noticed the same issue and it happens with version 12_1_4. We can see the logs reaching the apache, but it does not appear in the “visits log”. Below is the User-Agent that comes for this when accessed using the “Captive Network Assistant” page. However, it is working fine when used using Safari or Chrome browser.

Captive Network Assistant User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/16D57

Safari User-Agent: Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1"

Below is the User agent for Captive Network assistant for 12.3, it is being logged properly
“Mozilla/5.0 (iPhone; CPU iPhone OS 12_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148”

Hi @Riazuddin_Nazimuddin,

The user agent seems to be detected fine by DeviceDetector (apart from the odd 12_1_4 version)
https://devicedetector.lw1.at/Mozilla%2F5.0%20(iPhone;%20CPU%20iPhone%20OS%2012_1_4%20like%20Mac%20OS%20X)%20AppleWebKit%2F605.1.15%20(KHTML,%20like%20Gecko)%20Mobile%2F16D57

Do I understand it correctly that only visits in the Captive Portal overlay are not rendered correctly?

Maybe that browser has a special config regarding third-party-domains or has DNT enabled?

1 Like

Hi @Lukas

Yes, that is correct

The same browser works find with iOS version 10, and iOS versions 12.3 (Haven’t checked other iOS versions yet)

What I meant is that the Captive Portal browser in the past worked differently than the real safari on the same device, so I wouldn’t be surprised if there was a missing feature or bug in it that breaks your Matomo integration.

http://divideandconquer.se/2017/01/26/limitations-of-apple-ios-captive-portal-web-browser/

I could see that the post methods are being called in apache access_log. Could it be that it is not being parsed properly?