3.13 archive problem

Support & Bugs
1

We’re getting this message in the archive log a few times a day:

INFO [2019-12-03 20:05:04] 57741 - tracking data found for website id 8 since 2019-12-03 20:00:52 UTC (since the last successful archiving)
INFO [2019-12-03 20:05:04] 57741 Will pre-process for website id = 8, period = day, date = last2
INFO [2019-12-03 20:05:04] 57741 - pre-processing all visits
INFO [2019-12-03 20:10:01] 57741 Archived website id = 8, period = day, 0 segments, 21872 visits in last 2 days, 8357 visits today, Time elapsed: 296.438s
INFO [2019-12-03 20:10:01] 57741 Will pre-process for website id = 8, period = week, date = last2
INFO [2019-12-03 20:10:01] 57741 - pre-processing all visits
INFO [2019-12-03 20:14:57] 57741 Archived website id = 8, period = week, 0 segments, 77564 visits in last 2 weeks, 21950 visits this week, Time elapsed: 296.185s
INFO [2019-12-03 20:14:57] 57741 Will pre-process for website id = 8, period = month, date = last2
INFO [2019-12-03 20:14:57] 57741 - pre-processing all visits
INFO [2019-12-03 20:20:12] 57741 Archived website id = 8, period = month, 0 segments, 368920 visits in last 2 months, 29600 visits this month, Time elapsed: 315.206s
INFO [2019-12-03 20:20:12] 57741 Will pre-process for website id = 8, period = year, date = last2
INFO [2019-12-03 20:20:12] 57741 - pre-processing all visits
INFO [2019-12-03 20:25:48] 57741 Archived website id = 8, period = year, 0 segments, 637398 visits in last 2 years, 637398 visits this year, Time elapsed: 335.908s
WARNING [2019-12-03 20:25:48] 57741 /var/data/www/matomo/libs/Zend/Db/Statement/Mysqli.php(105): Warning - Error while sending STMT_CLOSE packet. PID=57741 - Matomo 3.13.0

It’s always when it tries to process the year when traffic overall is fairly high and happens on a few of teh larger sites. When the warning is issued, archiving stops until the next hourly run and other sites pick up from there. Server loads are all in the 20-30% range when it occurs. As you can see in the log, the failure occurs after the year tries to process for approx. 5 minutes. Is there a php setting that controls a timeout here? It seems like we may have a 5 minute setting somewhere based on the timings.

Here are MYSQL Settings from other threads I’ve researched:
wait_timeout: 720 secons (12 mins)
max_allowed_packet: 805306368

Thanks for any help.

2

Could someone from Matomo weigh in here please?

Still getting the error as collective traffic ramps up in the day. For a given site, is each period for a site’s processing managed independently or does it count as a group for the site? We’re trying to determine if we’re looking for a timeout or other limit that’s based on the 342 seconds for the year processing on site 8, or if all periods for site 8 count towards a potential limit.

Thanks.

INFO [2019-12-05 20:05:02] 92490 - tracking data found for website id 8 since 2019-12-05 20:02:41 UTC (since the last successful archiving)
INFO [2019-12-05 20:05:02] 92490 Will pre-process for website id = 8, period = day, date = last2
INFO [2019-12-05 20:05:02] 92490 - pre-processing all visits
INFO [2019-12-05 20:09:57] 92490 Archived website id = 8, period = day, 0 segments, 22232 visits in last 2 days, 8508 visits today, Time elapsed: 294.712s
INFO [2019-12-05 20:09:57] 92490 Will pre-process for website id = 8, period = week, date = last2
INFO [2019-12-05 20:09:57] 92490 - pre-processing all visits
INFO [2019-12-05 20:15:09] 92490 Archived website id = 8, period = week, 0 segments, 105220 visits in last 2 weeks, 49606 visits this week, Time elapsed: 312.022s
INFO [2019-12-05 20:15:09] 92490 Will pre-process for website id = 8, period = month, date = last2
INFO [2019-12-05 20:15:09] 92490 - pre-processing all visits
INFO [2019-12-05 20:20:27] 92490 Archived website id = 8, period = month, 0 segments, 396564 visits in last 2 months, 57244 visits this month, Time elapsed: 318.187s
INFO [2019-12-05 20:20:28] 92490 Will pre-process for website id = 8, period = year, date = last2
INFO [2019-12-05 20:20:28] 92490 - pre-processing all visits
INFO [2019-12-05 20:26:10] 92490 Archived website id = 8, period = year, 0 segments, 665047 visits in last 2 years, 665047 visits this year, Time elapsed: 342.995s
WARNING [2019-12-05 20:26:10] 92490 /var/data/www/matomo/libs/Zend/Db/Statement/Mysqli.php(105): Warning - Error while sending STMT_CLOSE packet. PID=92490 - Matomo$