Recently we enabled queued tracking (settings below).
Now between once per day or once per week “cron tab” console core:archive throw the following error:
ERROR [2019-12-01 12:00:09] 30416 Got invalid response from API request: ?module=API&method=API.get&idSite=1&period=day&date=last52&format=php&trigger=archivephp. Response was ‘a:2:{s:6:“result”;s:5:“error”;s:7:“message”;s:99:“Mysqli statement execute error : Deadlock found when trying to get lock; try restarting transaction”;}’
ERROR [2019-12-01 12:00:09] 30416 Empty or invalid response ‘’ for website id 1, Time elapsed: 1.635s, skipping
ERROR [2019-12-01 12:00:29] 30416 2 total errors during this script execution, please investigate and try and fix these errors.
Any idea?
Queued Tracking settings:
Settings that will be used:
Backend: mysql
NumQueueWorkers: 8
NumRequestsToProcess: 150
ProcessDuringTrackingRequest: 0
QueueEnabled: 1
Redis backend only settings (does not apply when using MySQL backend):
Host: 127.0.0.1
Port: 6379
Timeout: 0
Password:
Database: 0
UseSentinelBackend: 0
SentinelMasterName: mymaster
Version / stats:
PHP version: 7.3.12
Uname: Linux 3.12.18-clouder0 #1 SMP Tue Oct 9 18:14:47 EEST 2018 x86_64
Backend version: 5.6.36-82.1-log
Memory: array (
‘used_memory_human’ => ‘disabled’,
‘used_memory_peak_human’ => ‘disabled’,
)
Performing some tests:
Connection works in general
Initial expire seems to be set correctly
setIfNotExists works fine
expireIfKeyHasValue seems to work fine
Extending expire seems to be set correctly
expireIfKeyHasValue correctly expires only when the value is correct
Expire is still set which is correct
deleteIfKeyHasValue seems to work fine
List feature seems to work fine
we’d like to suggest to apply 2 sets of changes, then run the queued tracking process with a new parameter: -vvv
and let us know what is the output then?
Hello, I’m still on version 3.13.0, and since this version I get this error message every now and then (once a day / week):
Any idea or advise?
1)
Cron <asi@asi> /usr/local/php73/bin/php /home/asi/public_html/matomo/console core:archive --url=https://matomo.asi.asia/ > /dev/null
ERROR [2020-01-24 03:05:03] 15690 Got invalid response from API request: ?module=API&method=API.get&idSite=23&period=day&date=last2&format=php&trigger=archivephp. Response was 'a:2:{s:6:"result";s:5:"error";s:7:"message";s:99:"Mysqli statement execute error : Deadlock found when trying to get lock; try restarting transaction";}'
ERROR [2020-01-24 03:05:03] 15690 Empty or invalid response '' for website id 23, Time elapsed: 0.718s, skipping
ERROR [2020-01-24 03:05:17] 15690 2 total errors during this script execution, please investigate and try and fix these errors.
[Exception]
2 total errors during this script execution, please investigate and try and fix these errors.
core:archive [--url="..."] [--force-all-websites] [--force-all-periods[="..."]] [--force-timeout-for-periods[="..."]] [--skip-idsites[="..."]] [--skip-all-segments] [--force-idsites[="..."]] [--skip-segments-today] [--force-periods[="..."]] [--force-date-last-n="..."] [--force-date-range[="..."]] [--force-idsegments="..."] [--concurrent-requests-per-website[="..."]] [--concurrent-archivers[="..."]] [--disable-scheduled-tasks] [--accept-invalid-ssl-certificate] [--php-cli-options[="..."]]
2)
Cron <asi@asi> /usr/local/php73/bin/php /home/asi/public_html/matomo/console core:archive --url=https://matomo.asi.asia/ > /dev/null
ERROR [2020-01-23 00:35:03] 21953 Got invalid response from API request: ?module=API&method=API.get&idSite=10&period=day&date=last2&format=php&trigger=archivephp. Response was 'a:2:{s:6:"result";s:5:"error";s:7:"message";s:99:"Mysqli statement execute error : Deadlock found when trying to get lock; try restarting transaction";}'
ERROR [2020-01-23 00:35:03] 21953 Empty or invalid response '' for website id 10, Time elapsed: 1.057s, skipping
ERROR [2020-01-23 00:35:11] 21953 2 total errors during this script execution, please investigate and try and fix these errors.
[Exception]
2 total errors during this script execution, please investigate and try and fix these errors.
core:archive [--url="..."] [--force-all-websites] [--force-all-periods[="..."]] [--force-timeout-for-periods[="..."]] [--skip-idsites[="..."]] [--skip-all-segments] [--force-idsites[="..."]] [--skip-segments-today] [--force-periods[="..."]] [--force-date-last-n="..."] [--force-date-range[="..."]] [--force-idsegments="..."] [--concurrent-requests-per-website[="..."]] [--concurrent-archivers[="..."]] [--disable-scheduled-tasks] [--accept-invalid-ssl-certificate] [--php-cli-options[="..."]]
I’m seeing it too on 1.13.1. As a workaround I disabled processing during tracking requests in QueuedTracking and added a lock file to cron tasks for archiving and queue processing so that they never run simultaneously. Not quite optimal since archiving can take a good while with the number of sites we have, but avoids the locking issues nevertheless.
Here is an example that works with Linux. The first line runs archiving every half hour (15 minutes past and 45 minutes past the hour) between 03:00 and 23:00 (we run backups and log imports between 23:00 and 03:00 so just skip archiving during those). It waits to acquire the lock for five minutes and gives up if it can’t acquire it. The second line runs queue processing every minute and waits 30 seconds to acquire the same lock as the first one. Note that this will also prevent multiple queue processes from running in parallel, so the configuration for number of queue workers doesn’t make a difference.
The second line so long only because it includes commands to write timestamps in the log file. I find the log pretty useless if it doesn’t include any timing information.
