How to abort --force-all-periods in archiving? it takes impossible long!


#1

Hello,

i had a error in the .js tracking code and tried to import from my log files. I imported just 1 day of logs (~5k hits).
I followed How do I reprocess all websites, all dates and all periods, after initial import of logs? - Analytics Platform - Matomo and set the date exactly as in the faq.

Now he is trying to archiv every site with every period for years back in time :frowning:
After one day work he aborts:


INFO CoreConsole[2015-01-27 16:23:31] Archived website id = 2, period = year, 2327969 visits in last last6 years, 6883 visits this year, Time elapsed: 95772.945s
ERROR CoreConsole[2015-01-27 16:23:31] SQLSTATE[HY000]: General error: 2006 MySQL server has gone away

How can is set this “working period” back to just a few days?

Also 2015_blob is giant?!

Thanks for any help!
Greetings - max


(Matthieu Aubry) #2

MySQL server has gone away

see: How do I fix the error "Mysql Server has gone away"? - Analytics Platform - Matomo


#3

Its been a while…

After weeks i managed to get one run complete. My SQL instance is stable now and i tried everything i can image.

Archiving took more than 7 days for one site and period year. It took one week again to achive the same site a few days later. It is impossible to archive it completly again. It took over two weeks for a full archiving :frowning:

My Database has now grown >200GB, what normally would be around 1 Billion views. I host a few small sites, that is impossible! (see How to configure Piwik for speed - Analytics Platform - Matomo)

http://i.imgur.com/oV3C3qOl.png

What is wrong here? :frowning:


(Matthieu Aubry) #4

Hi Max

please try upgrade to latest version then run: ./console core:purge-old-archive-data

it should fix the issue, if not, please open a bug report at: GitHub - matomo-org/piwik: Liberating Web Analytics. Star us on Github? +1. Matomo is the leading open alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites, apps & the IoT and visualise this data and extract insights. Privacy is built-in. We love Pull Requests!


#5

hello matt,

i fixed(?) it before. I dropped all piwik_archive tables and made a complete run on a freshly installed machine with the newest versions of all. it took a while but completed, and now processing is just taking about 8 minutes every hour.
Also the database shrink a lot:

I hope the next time the new command will help me / others :slight_smile: