Way to much data and matomo Cron tab doesn't have enough memory

[Snorre_Jablonski]

I have several issues regarding data on my Matomo installation. With about 30 000 visitors daily on an onprem internal network website my MySQL database has now passed 15 GB worth of data. The last 10GB has come in a bit over 3 weeks after anonymization was turned on. Anyone with similar issues or any ideas on why this happens?

Second is my Matomo Cron tab which runs as a sidecar container in a Linux based onprem K8s platform. I have set the memory to “8G”, but when running the manual console-command for archiving inside the container I’m told that memory is exhausted and it needs aprox “32G”. This is way above the limit for my K8s cluster and seems way too high. Again, is this because of anonymization or some other screw-up on my side in setting up Matomo?

All responses are highly appreciated and I am also capable of supplying more information about the setup.

[heurteph-ei (Philippe)]

@Snorre_Jablonski

I have several issues regarding data on my Matomo installation. With about 30 000 visitors daily on an onprem internal network website my MySQL database has now passed 15 GB worth of data. The last 10GB has come in a bit over 3 weeks after anonymization was turned on. Anyone with similar issues or any ideas on why this happens?

Did you enable the DBStats core plugin?

@Snorre_Jablonski

Second is my Matomo Cron tab which runs as a sidecar container in a Linux based onprem K8s platform. I have set the memory to “8G”, but when running the manual console-command for archiving inside the container I’m told that memory is exhausted and it needs aprox “32G”. This is way above the limit for my K8s cluster and seems way too high. Again, is this because of anonymization or some other screw-up on my side in setting up Matomo?

Personally, I run the archive CRON (and all thing that is not directly tracking) on a separate server (then I have 2 servers for tracking and one for archive-CRON + management +…)
Also I configured:

–php-cli-options="-d memory_limit=3G"

See:

Just cannot get archiving to work

Hi @trackymctrackface, you have 4GB available for your plan but its possible you aren’t actually using 4GB for your archiving cron. Can you try setting a specific memory limit for the archiver using this option: --php-cli-options="-d memory_limit=3G" This will set the memory limit for the archiving process to 3GB. If your web server doesn’t support managing processes via cli - this would mean it will still use the web request limit. Can you also provide us with the output of: ./console diag…