I’m currently running an A/B test for two different versions of our website’s homepage. Each visitor to the website has a 50% chance of seeing either version. This randomness is set up with the A/B testing plugin.
At the moment, it seems like the A/B testing plugin is consistently favoring one version over the other. Right now, 29% of visitors are being directed to the original homepage, while 71% are seeing the second version. This is far from desirable.
This isn’t the first time this has happened; the same pattern occurred in a previous test.
Could you please help me understand:
How the plugin decides which version to show to visitors?
A/B test redirects to random variations. Difference will reduce when the test is nearing its end and it will even out once the test is ended as per your configuration. You may see a difference when the test is in progress and it is an expected behaviour.
Basically, every visitor to the website starts with the original version, and it’s redirecting correctly. I can see this in the visitor log. However, there’s no record of people entering the second homepage in the A/B testing tool because I set a rule that excludes any visitor who enters the “newhomepage” site from being included in the experiment.
As a PhD student, I’m running a test on this tool before my main research begins. I’ve spent over a month figuring out how it works. Can I get help from someone on your team via a video call to set up my experiment correctly and move forward on this?
I haven’t gotten a response from your team. I only got an automated response saying the A/B testing tool randomizes automatically, as they sent before.
We need guidance from a team member at Matomo on this matter, as it is urgent for my research work.
@Emerson_N I spoke with someone from your team. Thank you.
Everything is now sorted out.
The problem was with the placement of the code on our site.
I had placed the A/B test code before the page view tracking code, but your colleague explained that it should go below the initial site tracking code. This mistake might be due to unclear instructions during the A/B test setup.
Clearer instructions or a helpful GIF/resource on this would prevent such issues in the future.
There is an initial Matomo tracking code that tracks visits. For the A/B testing, I had to place the A/B test code provided below the tracking code for visits in the “” section of your code.
Example:
<head>
<!-- Matomo Tracking Code -->
<script type="text/javascript">
var _paq = window._paq = window._paq || [];
/* tracker methods like "setCustomDimension" should be called before "trackPageView" */
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);
(function() {
var u="//your-matomo-domain/";
_paq.push(['setTrackerUrl', u+'matomo.php']);
_paq.push(['setSiteId', 'YOUR_SITE_ID']);
var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
g.type='text/javascript'; g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
})();
</script>
<!-- A/B Testing Code -->
<script type="text/javascript">
// Your A/B testing code here
</script>
</head>
