Questions regarding Piwik servers with a large number of sites

My company’s Piwik server currently has 15,187 sites of various traffic quantities (though, none likely to be very high in page-views-per-day sense).

Our analytics reporting applications have problems displaying data of the current month. The API call for actions/pageviews/etc will return a null set IF the archiver hasn’t been run for a site the day of the request. Once the site’s data has been archived that day, everything’s fine.

So question 1, is the aforementioned a known Piwik bug?

Question 2:

The problem is, the archiver (archive.php) is taking roughly 30 hours to archive all 15k websites. Spawning 2 instances of the archiver concurrently reduced that ETA to 25 hours, but we’d ideally like to see it and 6 hours or less.

Any answers and pointing in the right direction would be much appreciated. Thank you for your time.

Dan

Hi Dan,

The best page to look is: Optimize and Scale Piwik - Analytics Platform - Matomo

But in your case I would highly recommend to contact us for professional support at: http://piwik.org/consulting/#contact-consultant