I’m currently running an A/B test for two different versions of our website’s homepage. Each visitor to the website has a 50% chance of seeing either version. This randomness is set up with the A/B testing plugin.
At the moment, it seems like the A/B testing plugin is consistently favoring one version over the other. Right now, 29% of visitors are being directed to the original homepage, while 71% are seeing the second version. This is far from desirable.
This isn’t the first time this has happened; the same pattern occurred in a previous test.
Could you please help me understand:
How the plugin decides which version to show to visitors?
A/B test redirects to random variations. Difference will reduce when the test is nearing its end and it will even out once the test is ended as per your configuration. You may see a difference when the test is in progress and it is an expected behaviour.
Basically, every visitor to the website starts with the original version, and it’s redirecting correctly. I can see this in the visitor log. However, there’s no record of people entering the second homepage in the A/B testing tool because I set a rule that excludes any visitor who enters the “newhomepage” site from being included in the experiment.
As a PhD student, I’m running a test on this tool before my main research begins. I’ve spent over a month figuring out how it works. Can I get help from someone on your team via a video call to set up my experiment correctly and move forward on this?
I haven’t gotten a response from your team. I only got an automated response saying the A/B testing tool randomizes automatically, as they sent before.
We need guidance from a team member at Matomo on this matter, as it is urgent for my research work.
