utrautma
October 27, 2023, 2:46pm
1
To set the attribution for a converted goal to the referrer of the first visit , I would like to use the function setConversionAttributionFirstReferrer .
At least that’s how I understood how it works, even if the documentation (here or here FAQ ) isn’t very precise in this regard. But in this old issue it sounds like I want it.
I built a simple test setup to see why it doesn’t work in a more complex environment, but I notice that the referrer of the first visit is not used for the converted goal or only in a few cases.
_paq.push(['setConversionAttributionFirstReferrer',true]);
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);
(function() {
var u="//localhost/matomo/";
_paq.push(['setTrackerUrl', u+'matomo.php']);
_paq.push(['setSiteId', '1']);
Here some findings:
First visit : Referrer from other website, Recent visit with conversion: Referrer from campaign → Result: Attribution goes to campaign - wrongFirst visit : Referrer from campaign, Recent visit with conversion: Referrer from other campaign–> Result: Attribution goes to recent campaign - wrongFirst visit : Referrer from campaign, Recent visit with conversion: Referrer from other website–> Result: Attribution goes to campaign - rightFirst visit : Referrer from website, Recent visit with conversion: Referrer from other website–> Result: Attribution goes recent website - wrong
I know this is all a little complicated. But maybe someone here has experience with this function.
Here are some screenshots:
Hi @utrautma setConversionAttributionFirstReferrer works thanks to cookie (the cookie stores the first referrer and I suppose Matomo scripts provides this referrer with further requests). Do you track with cookie?
opened 11:57AM - 14 Jul 23 UTC
Bug
Hi DevTeam!
I have gathered some questions and thoughts about the Matomo Attr… ibution Model. I have tried to explain it as best as I can, but I’m not sure if I’m correct. So please correct me if I’m wrong.
First we have the questions from a client, then I have added my thoughts and findings below the Results headline.
_From client:_
## MATOMO ATTRIBUTION RULES
1. Visits are last touch
https://matomo.org/faq/troubleshooting/faq_50/
> Higher “Direct Entry” visits in Matomo When tracking the acquisition source of visitors, Google Analytics stores and uses campaign data for up to 6 months and attributes subsequent direct entry visits to the original campaign acquisition source. (So Google Analytics will report a higher number of visits attributed to “Campaigns” because these visits used a campaign in the previous 6 months.) Whereas Matomo tracks any new “Direct Entry” visits as direct entries and does not attribute these new “Direct Entry” visits to their original acquisition source. With Matomo’s Multi Channel Conversion Attribution plugin, you can apply different attribution models to your goal conversions.
2. Conversions are last non-direct touch
https://matomo.org/faq/general/what-is-the-default-attribution-model-used-in-matomo/
## Data that support/ contradicts these rules
1. Our Direct visits data is significantly higher than GA. All data that we have supports that this rule is correct
2. Our conversion data does NOT support this
What we can see is that:
* A. CHANNEL TYPES do report on a last non-direct touch basis
* B. ANY CAMPAIGN PARAMETER (Source | Medium | Contents | Names) do not report on a last non-direct touch basis.. they are LAST TOUCH
### Why? How?
In the Matomo reporting interface:
- the Channel type report uses the API method "`Referrers.getReferrerType`".
- The campaign parameter type reports use methods such as: "`MarketingCampaignsReporting.getSource`" & "`MarketingCampaignsReporting.getMedium`"
Having examined the contents of these respective files, specifically looking for fields related to campaign activity, I can observe the following:
* `Referrers.getReferrerType` contains **ONLY** Label (which is the Campaign Name)
* `MarketingCampaignsReporting.getSource` contains **ONLY** Source
* `MarketingCampaignsReporting.getMedium` contains **ONLY** Medium
What I assume is happening is:
* **#A1**: the `Referrers.getReferrerType` is correct (it is last non direct touch) - it allows for a count of the total `ecommerceOrders` for each campaign. I assume that this data is written to Matomo Using the values contained in the `_pk_ref` cookie because having examined the contents of that cookie, the ONLY campaign information that is present is the **Campaign NAME**
* **#A2**: The `MarketingCampaignsReporting.*` queries do not use the `_pk_ref` cookie because they can't. It does not contain additional campaign parameters. These reports can only use what it has available which is essentially data from the Visits Log and that is **LAST TOUCH** (see `#1` above)
## Re-cap:
Our aim is to get:
* Campaign reports (Campaign Names
* Campaign Keywords
* Campaign Sources
* Campaign Mediums
* Campaign Contents
* Campaign Source - Medium
* Campaign Ids
* Campaign Groups
* Campaign Placements
* for orders on a last non direct touch basis.
A potential solution would be to extend the contents of the `_pk_ref` cookie to include these parameters and write them to either the `Referrers.getReferrerType` data set (or a new data set) then subsequently update the respective `MarketingCampaignsReporting.*` data sets to refer to this data.
## Tasks
1. Conduct a set-through analysis of the Matomo SQL queries to confirm/ deny **#A1** and **#A2**
2. Propose & build a solution that meets our aim
---
# Results
After thorough analysis of the SQL queries, I have come to the following conclusions:
## #A1
* The API call `Referrers.getReferrerType`, runs about 30+ queries in the DB. (I have stored all Query logs and attached them in a zip-file).
* Most of the Queries are pretty much standard, like security checks or look ups.
* The interesting Query is fetching data from an archive blob data table.
* Meaning, that the calculation is already done, with core:archiving
* Following that trail takes me to the Referrers plugin, which is a default plugin I think.
* The code base is found here: https://github.com/matomo-org/matomo/blob/f226904b1ef360c35f2e42dbc685f5db3d41a1fb/plugins/Referrers/Archiver.php#L84
* In the Referrers plugin I found the function which gives a goal its attribution.
* You can find it here: https://github.com/matomo-org/matomo/blob/5.x-dev/plugins/Referrers/Columns/Base.php#L612C34-L612C34
* Some interesting notes in that function are:
>>>
Attributing the correct Referrer to this conversion.
Priority order is as follows:
0) In some cases, the campaign is not passed from the JS so we look it up from the current visit
1) Campaign name/kwd parsed in the JS
2) Referrer URL stored in the _ref cookie
3) If no info from the cookie, attribute to the current visit referrer
>>>
* I also checked the JS tracker, and there is functionality there that tries to detect and attribute Referrer information, based on the _ref cookie, check it out:
* https://github.com/matomo-org/matomo/blob/5.x-dev/js/piwik.js#L3798
* Some interesting notes:
>>>
cookie 'ses' was not found: we consider this the start of a 'session'
Detect the campaign information from the current URL
Only if campaign wasn't previously set
Or if it was set but we must attribute to the most recent one
Note: we are working on the currentUrl before purify() since we can parse the campaign parameters in the hash tag
>>>
* It sets some values into the cookie, such as:
* `campaignNameDetected` // Name if detected
* `campaignKeywordDetected` // Keyword if detected
* `referralTs` // Current time the user visits the site
* `referralUrl`
* And some interesting notes:
>>>
Store the referrer URL and time in the cookie;
referral URL depends on the first or last referrer attribution
>>>
* The cookie is set for 6 months
* So in theory, The cookie should be respected and the conversion should be attributed to the first referrer if I’m not mistaken, it also seems to depend on some kind of setting/config.
* `configConversionAttributionFirstReferrer`
* Which I currently can’t find in the value for.
* I also checked the Goal plugin. Which generates that Channel Type View. See file `Goals-Archiver.sql`.
* Ecommerce —> Sales —> Channel Type
* The Query that build that report is looking into the `log_conversion` table, meaning that it shouldn’t be able to know anything about Multi Channel Attribution, because all the information is already in that table.
* `referer_keyword`
* `referer_name`
* `referer_type`
* `campaign_content`
* `campaign_group`
* `campaign_id`
* `campaign_keyword`
* `campaign_medium`
* `campaign_name`
* `campaign_placement`
* `campaign_source`
* And if you look at the Campaign Mediums - See file: `MarketingCampaignsReporting-Archiver.sql`
* Ecommerce —> Sales —> Campaign Mediums
* It is aggregating it’s report from `matomo_log_visit` and `log_conversion`
* So I’m not quite sure how it’s suppose to work.
* But potentially these reports are not respecting the last non direct touch approach.
* I also added logs from MultiChannelConversionAttribution-Archiver.sql
* It is joining multiple tables in the queries.
These queries are very complex and hard to fully understand if one isn’t familiar with the datasets. And it’s too soon for me to assess if it’s supposed to be like this, if the reports are correct or not. It could be related to configuration, as I’ve learned reading the code now, there is that `configConversionAttributionFirstReferrer` setting for an example.
In the file called `customer-journey.sql`
I reproduced the steps we did together.
1. I enter the site from a campaign URL.
2. I browse around.
3. I delete the `_ses` Cookie and close the tab.
4. I enter directly to the site and finish an order, which should convert.
And I can see in the last query that it attributes to my campaign. Which is the correct behavior, right? Last non direct touch.
# #A2
The MarketingCampaignsReporting plugin uses the logs, but the logs are updated with the referral data. So it should be the same behavior. But that doesn’t exclude the possibility of the query(ies) to be wrongly formatted.
What I mean is that the referrer data is the same, it is there. But for some reason, it is not respecting the last non touch approach.
## Conclusion:
We need Matomos Core Team to help us to fully understand which report is correct and how we can get the data we need.
Could it be that the campaign info is missing from the inserts in:
https://github.com/matomo-org/matomo/blob/115527353a9e75e01aa4d263408956ae45403bea/plugins/Referrers/Archiver.php#L61
[queries.zip](https://github.com/matomo-org/matomo/files/12050226/queries.zip)
opened 11:39AM - 22 Aug 22 UTC
Enhancement
This is a follow-up issue to the discussion in #18612
### Summary
When a v… isitor visits the page, the visit will be attributed to a certain referrer. This can currently be `website`, `campaign`, `search engine` or `social network`. If non of this referrer can be detected the visit will be attributed as `direct`.
When a visitor afterwards revisits the page coming from another referrer and the referrer was previously attributed as `direct`, we currently may update the attribution to this new referrer. See workflow below:
### Current Attribution workflow
### Conclusion
Updating the referrer in that case might not be correct, as the visitor initially came directly to the page, but would then be attributed to another channel.
As far as I know the code to update the referrer was initially added to fix some issue, where the referrer was not yet set with the first tracking request, but was sent with a later one. Not updating the referrer in that case, would then mean incorrectly attributing the visit as direct, instead of the correct referrer.
A possible solution could be to only set/update the referrer until the very first page view tracking request of a visit. This would solve the issue, that other tracking requests might have already created a visit without a referrer, as it would then be updated with the page view request.
Changing any behavior in referrer attribution might result in differently tracked data and thus changing numbers. As this might bring trouble, we should not change that outside of a major release.
Also we should consider to do that using a config flag, that will only be activated for new installs automatically, so it won't harm any existing install by default.
At last, there is a (premium) plugin that can change the attribution rules:
utrautma
October 31, 2023, 3:10pm
3
Hi @heurteph-ei ,
Thank you for finding the issues.
I’ll investigate this further, but from what it looks like at first glance, the attributions could be as shown in the graphic above.
1 Like
