Multiple instances of archive cron?


#1

We have a website with > 10 mio visits per day. I split the archive cron into three parts for running daily, weekly and monthly statistics to increase speed.

The crons runs every 5 minutes and check, if a process is running to avoid multiple instances:

/usr/bin/php5 /opt/app/piwik/console core:archive --disable-scheduled-tasks --url=DOMAIN --force-idsites=1 --force-periods=day
/usr/bin/php5 /opt/app/piwik/console core:archive --disable-scheduled-tasks --url=DOMAIN --force-idsites=1 --force-periods=week
/usr/bin/php5 /opt/app/piwik/console core:archive --disable-scheduled-tasks --url=DOMAIN --force-idsites=1 --force-periods=month

The daily cron needs at the afternoon up to 200 minutes to complete the task.

Is it possible to run multiple archive jobs for the same ID to increase the update rate?


(Matthieu Aubry) #2

Not yet but maybe this ticket will help you: Improve multi threading in archiving to be more linear · Issue #5363 · matomo-org/matomo · GitHub