ABTesting phplibrary - Test definitions in Matomo ignored?

Premium Features
1

hi
We started using AB Testing and to be honest, we are quite frustrated about the really poor documentation on technical integration e.g. with PHP Library.

I describe you here our setup and requirements, as well what we have done so far.

Setup & Requirements

We are using Drupal (which is based on PHP).

For AB-Tests we will use two different URL’s.

We want to have a very easy setup for our marketer for AB Tests he defines. Easy setup means:

  • For a new test it should not be required to add custom JS Code or whatever code in Website Pages
  • IF it is still required, we need a very easy way where the marketer can enter custom code in drupal and it is loaded on the pages (we can implement it)

What we tried first: JS Setup

The problems:

1: with JS the original layout/page is always loaded first and sometimes get redirected to the next variant

The issue here is the user can see the original layout for a short time before redirecting, after doing some research we can solve this with css and js so the page is only displayed or visible after the page is redirected or loaded

→ Can you confirm this? Is there some documentation about hat?

2: When the test is defined for URL1 and URL2, then ULRS cannot be changed without a JS Code update.

This means: Each URL Change requires a JS Code update.

→ Is this correct?

3: Even when the distribution between two pages changes, e.g from 50/50 to 70/30 a JS Code update is needed?

→ Is this correct? Is there some documentation in which cases the JS Code must be updated?

4: When we exclude pages, add or remove variations or add a condition, or uncheck forward argument checkbox: Then always a JS Code update is needed?

→ Is this correct?

The alternative solution: Your PHP Library

Based on your documentation https://developer.matomo.org/guides/ab-tests/server we then decided to implement the PHP Library.

After 1.5 days implementation we found now out, that we must also define in Drupal for each test the following:

  • AB Name / Experiment Name
  • URL Page for AB Testing
  • variations 1…n: name, redirect, allocation

And all this for each experiment.

It seems that all what is in Matomo defined ( and used in the JS variant) is ignored in the PHP LIbrary variant.

Is this true? There is no way to access experiemnts in Matomo from phplibrary and each experiment must be setup twice in Matomo and PHP?

thanks for your feedback

2

Hi @regardtrack ,

A reply was sent to your support request already from our support channel.

4

hi @karthik
I saw the mail, you answered 1(!) of the questions above.
The first part of questions is just for confrimation, to make sure we understood correctly how Matomo AB Testing works.

However, as you can see the main question is about the ABTESTING Phplibrary where we try now since 1 week to get an answer.

I write here again the question, so you dont have to search for it:
The alternative solution: Your PHP Library

Based on your documentation https://developer.matomo.org/guides/ab-tests/server we then decided to implement the PHP Library.

After 1.5 days implementation we found now out, that we must also define in Drupal for each test the following:

AB Name / Experiment Name
URL Page for AB Testing
variations 1…n: name, redirect, allocation

And all this for each experiment.

It seems that all what is in Matomo defined ( and used in the JS variant) is ignored in the PHP LIbrary variant.

Is this true? There is no way to access experiemnts in Matomo from phplibrary and each experiment must be setup twice in Matomo and PHP?

5

@karthik hi, unfortunately we still have no answer on this question we asked you one week ago.

6

Hi @regardtrack. You are correct. The Innocraft/PHP Experiments framework does not collect AB Test Information from your Matomo instance so the AB Test would need to be configured in both the server-side script and in the Matomo UI.

If you have many AB Tests you could potentially use data from the AbTesting module of the reporting API to retrieve test information to automate your server-side test setup using custom script.

7

hi @Joshua123
thanks for the feedback.
Well, the purpose and motivation of using an AB Testing tool is exactly that there are many AB tests… Otherwise we would do it directly in our CMS.

Yes, reporting API maybe is an option, however it is time consuming to integrate it having the fact there is a PHP Library.

Therefore in short: This php library is not really of use, because we want to choose an ABTesting tool which makes work more productive

Do you have any plans to update phplibrary in the near future in order to read experiment data from matomo?

8

Hi @regardtrack ,
I made a feature request, internally, with the title

“Improve serverside (PHP) AB Testing in order to read experiment data from Matomo”

If you (and others who read this!) reply here with a brief wish list of specific things I will append them to that internal request.

1 Like
9

thanks, thats great
Overall the requirement is “simple”: When using an AB Testing tool, we want to define the experiments in one place. this means: when using the php library, it should read the experiments from matomo, without the need configure them again in php

10

Hi @regardtrack
Excellent. I appended that to the (internal) ticket. Thx

11

Adding some answers to the original post, by the way:

2: Yes
3: No that is set in the Matomo interface. Go to
Matomo >> A/B Tests >> Edit test >> Traffic allocation
4. Most of those require JS code changes in your page, yes. However redirects can be set in the Matomo interface.

12

1: This issue (the original page loading before the redirect) is common if you do not include the JS high enough in your code, like toward the top of the <head> element.