I am trying to exclude Headless agents from my tracking. This has worked in the past but for some reason these visits started slipping through the filters, spoiling my stats.
My Excluded User Agents configuration:
Example visit slipping through this config:
[I am only allowed to upload one image as new user, please imagine a Headless Chrome visit here]
Any idea why this happens?
I wonder whether I am not seeing a side effect of this:
Wow, this forum is really dead. No idea why I am answering to myself all the time, but maybe this will save somebody some time.
Today I learned there is a “Tracking Spam Prevention” Plugin for Matomo which does what I want, exclude headless browsers (amongst other things of course).
This plugin seems to have the exact same problem as I do. There is a gitlab issue which I am not allowed to paste here (no links allowed). Just look it up on the github repo of the above plugin.
It’s not possible to filter all bots and similar.
Use the Matomo Plugin “Tracking Spam Prevention” ore use PHP in your website.
PHP 8
$remote_useragent = $_SERVER['HTTP_USER_AGENT'];
$headless_useragent_array = [
'HeadlessChrome',
'PhantomJS',
'Electron',
'ApacheBench',
'Siege/',
'https://k6.io/',
'Radview',
'Locust',
'Cypress'
];
foreach ($headless_useragent_array as $headless_useragent_name) {
if (str_contains($remote_useragent, $headless_useragent_name) === false) {
// load the Matomo Tracking Script
untested.
Consider: Empty User Agents not included in this filter. For empty User Agents:
if (empty($remote_useragent) === true) {
PS: This is a user helps user community.
Thanks for your answer, I appreciate it. As I wrote above, I also discovered the plugin today. But the problem with Headless Chromes slipping through filters affects also the plugin.
Chrome seems to have moved its info whether it is headless or not to a different place (Client Hints), so currently there is a gap in all standard Matomo filter mechanisms to detect these.
I am not a PHP dev, but as far as I see your code above also only checks for user agent header and will not detect these.
I am tracking the github issue for this on the plugin repo, hoping there will be a new version soon. Then I’ll use the plugin.
When there is nothing to match this Headless Browser, then there is no way to detect it. You brought everything to match it. So, Chrome say, we give you nothing to match it. So, it is not possible to match it.
Not sure what you’re trying to say here. Chrome gives us some info to match, just in a different place than it was before. Here https://github.com/matomo-org/plugin-TrackingSpamPrevention/issues/108 is the issue which is exactly my use case, acknowledged by the Matomo team.
OK. Wait a minute, and then more infos on github.
Return back to the forum, because, this is also solved with JavaScript, and before Tracking will be activated.
Your Matomo URL String:
The User Agent has nothing information to detect the browser as an Headless Browser:
HTTP/1.1" 204 0 "https://www.example.com/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.6167.57 Safari/537.36"
So, it is not possible to detect the browser as an Headless Browser.
Inside the matomo.js are two bassically JavaScript functions are used and integraded in the the Matomo Tracking URL-Querystring as uadata.
Both don’t work in all browser.
navigator.userAgentData.brands
: https://developer.mozilla.org/en-US/docs/Web/API/Navigator/userAgentData
navigator.userAgentData.getHighEntropyValues.fullVersionList
: https://developer.mozilla.org/en-US/docs/Web/API/NavigatorUAData/getHighEntropyValues
By the second is this needed:
navigator.userAgentData.getHighEntropyValues.fullVersionList
Its an Array/Object, that are used as the value of the key uadata.
uadata={"fullVersionList":[{"brand":"Not A(Brand","version":"99.0.0.0"},{"brand":"HeadlessChrome","version":"121.0.6167.57"},{"brand":"Chromium","version":"121.0.6167.57"}],"mobile":false,"model":"","platform":"Linux","platformVersion":"5.15.0"}
So, yet we need a JavaScript code that detect Headless inside this datas. Further, a check is needed of the exist of this functions.
// Edit: deleted, because wrong code (:mebalo)
// new code will be soon.
untested.
Here a possible solution to detect Headless Chrome Browser: