Subsequent requests from same User ID not appearing in user ID visits log

I’m experiencing what I feel to be a bug. Concrete description of what I’m doing for a customer:

The customer launched a marketing campaign targeting users to book an event over a landingpage. The sequence of events is:

  • Users receive an e-mail
  • click on a button on it
  • arrive on the landingpage > this should be tracked, event (A)
  • click on a button on that landingpage > this should be tracked, event (B)
  • which then redirects them to a calendly booking page, where they perform the booking > the performed booking should be tracked, event (C).

Mission: Have to track the entire event sequence A-C.

I cannot control the booking integration (I know it would be much better for the tracking of the entire event sequence to have the booking system integrated on the LP itself). Redirects after the booking process back to the LP are also not allowed.

I’m using Calendly’s Webhooks feature to forward the event of an booking through Calendly to Matomo. This works, but there is one problem with it:

  • Due to the problem mentioned here, we’ve set our configs to:
[Tracker]
enable_userid_overwrites_visitorid = 0

This successfully merges actions of an unidentified visitor to the ones of the same visitor if a user ID is assigned to later actions of the same visitor for the same visit.

But, with this config (Matomo 5.0.3), we have the problem that, when tracking (C) via webhook endpoint configured for calendly, Matomo initiates a completely new visit. Even if (A) and (B) are performed just a few seconds before (C), Matomo creates a completely new visit for the event (C).

If it would only be that problem, that’s fine, but the weird thing is that event (C) does not appear in the visitor profile of the identified user. Even if all the events (A), (B) and (C) are submitted with the cid parameter set to the same value, they are not merged into the visitor profile of the user of that cid!
I guess this must be a bug?

We’ve even programmed to omit the cip URL parameter in the tracking request URL that forwards the event (C) from Calendly to Matomo, as it was anyway not useful to submit the IP address of Calendly’s Webhook server as the client IP of our tracked request. But this did not change anything; the event (C) is still no part of the actions of the visitor profile for the concerned identified user. This makes it extremely frustrating to track the last event (C) of this event sequence. Although we track all three events with the same user ID (cid), the event (C) does not appear in the visits view of the visitor profile (Visits > Visitor Log > top right link saying “check visitor profile”, which should list all visits of the user of that user ID, but for the event (C) exclusively lists one visit (#1) with only the Calendly action tracked via webhook).

If this is not a bug, why is this happening, and how can we properly include the event (C) into the visitor profile of the concerned user?

Hi @fulstadev
How did you configure Matomo for cross-site tracking? (As I presume calendly and your landing page are not hosted on the same website…)

Hi @heurteph-ei we once had that setup according to your docs explaining about how to do it for js, e.g. (as we have multiple different landingpages pointing to the calendly booking page):

_paq.push(["setDomains", ["*.mylpdomain.com", "*.mysecondlpdomain.com"]]);
_paq.push(["enableCrossDomainLinking"]);

But we’ve no longer enabled it now, as we didn’t see the difference in doing that vs adding the concerned domain names in the URLs you can specify to let matomo know from which URLs events should be tracked, in the Matomo Settings. From what I understand now, the advantage would be that the visitor ID would be forwarded when getting from mylpdomain.com to the mysecondlpdomain.com, and then matomo.js would automatically read the visitor ID out of the URL, to in this way aggregate that action from tracked page A to tracked page B as a single visit, correct?

You are correct that calendly is hosted on a separate domain, and I have not included calendly’s domain within the script above. Still, I don’t see why I should do that, as we’re not integrating calendly via matomo.js but via webhook, as it is not possible to add custom js to our calendly booking page. Hence forwarding the visitor ID via URL to our calendly page, via cross domain tracking being enabled, would not be of any help in this regard, as I see no way of re-using that visitor ID forwarded to the calendly page, as the webhook action gets triggered in a separate environment, from the calendly servers. So to me it appears that the main issue is that matomo does not consider two events which are very close to each other (clicking on button leading to calendly on LP + booking appointment on calendly) and tied to the same user ID to be part of the same visit…?

Hi @fulstadev
Did you add also calendly to the domains list (point 1 of Matomo Analytics Platform FAQ):

  1. Configure your domain names as Alias URLs for your Matomo website. Login to Matomo and click on Administration > Websites > Manage. Edit your website, and specify all your domain names in the Alias URLs field. There must be two or more domains for cross domain to work.

This is probably mandatory for the visitor ID to be reused…

No I did not, but the event forwarded from the Calendly webhook into Matomo flows in into Matomo, it’s there. It’s just showing up in a completely isolated visit of the user of the concerned user ID, although the user of that user ID has other actions seconds before; that is what feels weird to us.

We haven’t added the webhook URL from calendly into the list of URLs of our website, but as we’re not able to rely on matomo.js for the calendly webpage, we thought this is not required?

Hi @fulstadev
You mentioned user ID, I thought it was the same visitor ID… ?

As the dataconsolidation is done by Matomo server, maybe this is needed… To be tested!

Hi @heurteph-ei thanks in general for your very prompt replies, first of all!

You are absolutely correct, I made a mistake there in my first question where I mentioned that the cid parameter is the same, which is wrong. The uid is the same value (user ID), while the visitor ID cip is different among the visit on the website vs the visitor ID delivered via the calendly webhook server (which makes sense, as there’s no way for calendly’s webhook server to know the visitor ID).

We’ll give it a try then.

@heurteph-ei So we’ve tried it out and what we experienced is somewhat weird. We’d be glad if you could help us out as we’ve already spent hours on a possible workaround, and it’s still not working as expected.

What we’ve tested:

  • User A books an event with a visitor ID aaaaaaaaaaaaaaaa (simplified), through which he obtains a user ID 123.
  • User A visit our landingpage again, with a new visitor ID bbbbbbbbbbbbbbbb (simplified, different from previous one). No user ID available yet.
  • we extract the visitor ID while user A is on the LP via js, and add that as query parameter that calendly’s webhook automatically forwards, to the calendly links of our LP. Hence all of our calendly links on the page (e.g. https://calendly/our-page) become e.g. https://calendly/our-page?utm_term=bbbbbbbbbbbbbbbb.
  • user A clicks on it and arrives on the calendly page, and books an appointment.
  • calendly webhook is triggered, sending booking data to our server endpoint listening to it.
  • Our server logs show that the visitor ID bbbbbbbbbbbbbbbb has properly been forwarded from calendly to our servers; so its extraction via js and forwarding to the calendly webhook payload works as expected.
  • From our server, we then forward the event to matomo, by explicitly setting the visitor ID to the extracted / obtained value bbbbbbbbbbbbbbbb, and the user ID based on obtained personal data, e.g. the email address. Hence, we again get the user ID 123, as it’s the same user booking the request.
  • Our logs show that the event sent to matomo used the URL with the parameter values inserted as expected, e.g. :

https://example.matomo.cloud/matomo.php?idsite=1&rec=1&apiv=1&r=096813&uid=123&_idts=1720113903&cid=bbbbbbbbbbbbbbbb&e_c=CALENDLY&e_a=Appointment+Booked&e_n=webhook

Note that we’ve explicitly removed the cip from the URL to prevent any potential IP-related identity resolution (if that even happens).

And it’s still not working. In the Matomo dashboard, the booked event is tied to the visitor profile resulting from the very first event booking of the user 123, with the visitor ID aaaaaaaaaaaaaaaa. So it seems that when a user ID is provided in the request, the passed visitor ID is ignored? Is that intentional? That would still be problem, as it would not allow to match anonymous actions tied to a visitor ID X to identfied actions of a specific user ID with a different visitor ID???

We’ve also noticed that, if we program an automatic redirect to occur after the calendly booking, back to our LP, the page visit of our LP after that redirect is then again assigned to the previous visitor ID & the previous visit bbbbbbbbbbbbbbbb. So it really seems that the webhook event is perceived as a completely isolated visit in matomo, even if we explicitly pass the visitor ID of the previous page visit. This is clearly a bug?

Hi @fulstadev
Sorry, the user journey you described is very hard to understand…
Could you share simplified queries parameters sent to Matomo (ignore the server and path + site ID https://example.matomo.cloud/matomo.php? , as they don’t vary and also params like rec (recording), apiv (API version), r (random), _idts (time stamp) as they don’t participate in the journey identification :wink: ), with mention of the sender (user + website or server). And also the simplified result in Matomo visits log…
Please note that Matomo is mainly based on visitor ID, then if it varies (in your example, if changes from aaa to bbb), then Matomo will have difficulties to reconciliate visitor journey.
Do you use Matomo cloud, or is it your own installation?

Hi @heurteph-ei the problem was basically that, if a customer sends an event with a visitor ID aaa and a user ID 123, and then a little later a second event with a visitor ID bbb and the same user ID 123 (different visitor ID due to the page redirect occurring through LP - calendly booking page - LP), matomo failed to merge that into the same visit, although it’s actually the same user ID.

We’ve been talking to your support now, as our account is a cloud account, and the problem is solved: We had asked your cloud support to set the value of enable_userid_overwrites_visitorid to 0 some months ago, and double-checked with them if they really did that. Unfortunately, they informed us this week that they previously specified the wrong config section for this setting, indicating the ‘General’ section instead of the ‘Tracker’ section. So the setting was not applied.

Since the enable_userid_overwrites_visitorid has been set to 0 (non-default value), everything gets properly merged into a single visit.

1 Like