Archive (web cron) raises Invalid 'log_visit.config_device_type'

Hello everyone,

running the archive web cronjob on a high traffic website with Matomo 4.1.0 the process raises errors and seems not to finish successfully.

Here’s what happens when running this process manually inside the browser (for better overview of the main error message, I skipped the middle of the output):

INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] ---------------------------
INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] INIT
INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] Running Matomo 4.1.0 as Super User
INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] ---------------------------
INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] NOTES
INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] - Async process archiving not supported, using curl requests.
INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] - Reports for today will be processed at most every 1800 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO CoreAdminHome[2020-12-22 17:05:56 UTC] [6f5e3] - Archiving was last executed without error 6 Stunden 34 Minuten ago
INFO CoreAdminHome[2020-12-22 17:06:01 UTC] [6f5e3] ---------------------------
INFO CoreAdminHome[2020-12-22 17:06:01 UTC] [6f5e3] START
INFO CoreAdminHome[2020-12-22 17:06:01 UTC] [6f5e3] Starting Matomo reports archiving...
INFO CoreAdminHome[2020-12-22 17:06:06 UTC] [6f5e3] Start processing archives for site 1.
INFO CoreAdminHome[2020-12-22 17:06:06 UTC] [6f5e3] Will invalidate archived reports for today in site ID = 1's timezone (2020-12-22 00:00:00).
INFO CoreAdminHome[2020-12-22 17:06:06 UTC] [6f5e3] Will invalidate archived reports for yesterday in site ID = 1's timezone (2020-12-21 00:00:00).
...
INFO CoreAdminHome[2020-12-22 17:06:25 UTC] [6f5e3] Finished archiving for site 1, 22 API requests, Time elapsed: 19.322s [1 / 1 done]
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Done archiving!
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] ---------------------------
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] SUMMARY
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Processed 21 archives.
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Total API requests: 22
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] done: 22 req, 28988 ms, 6 errors.
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Time elapsed: 28.988s
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] ---------------------------
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] SCHEDULED TASKS
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Starting Scheduled tasks...
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Scheduler: executing task Piwik\Plugins\CustomJsTracker\Tasks.updateTracker...
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Scheduler: finished. Time elapsed: 0.001s
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.deleteLogData...
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Scheduler: finished. Time elapsed: 0.000s
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.anonymizePastData...
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Scheduler: finished. Time elapsed: 0.000s
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] done
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] ---------------------------
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] ---------------------------
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] SUMMARY OF ERRORS
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Error: Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=day&date=2020-12-22&format=json&segment=deviceType%253D%253Dtablet%252CdeviceType%253D%253Ddesktop&trigger=archivephp. Response was '{"result":"error","message":"Invalid 'log_visit.config_device_type' segment value tablet,deviceType==desktop"}'
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Error: Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=week&date=2020-12-21&format=json&segment=deviceType%253D%253Dtablet%252CdeviceType%253D%253Ddesktop&trigger=archivephp. Response was '{"result":"error","message":"Invalid 'log_visit.config_device_type' segment value tablet,deviceType==desktop"}'
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Error: Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=month&date=2020-12-01&format=json&segment=deviceType%253D%253Dtablet%252CdeviceType%253D%253Ddesktop&trigger=archivephp. Response was '{"result":"error","message":"Invalid 'log_visit.config_device_type' segment value tablet,deviceType==desktop"}'
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Error: Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=year&date=2020-01-01&format=json&trigger=archivephp. The response was empty. This usually means a server error. A solution to this error is generally to increase the value of 'memory_limit' in your php.ini file.  For more information and the error message please check your web server's error Log file. As this core:archive command triggers PHP processes over HTTP, you can find the error message in your Matomo's web server error logs.
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Error: Error unserializing the following response from ?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=year&date=2020-01-01&format=json&trigger=archivephp: ''
INFO CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] Error: Got invalid response from API request: ?module=API&method=CoreAdminHome.archiveReports&idSite=1&period=year&date=2020-01-01&format=json&segment=deviceType%253D%253Dtablet%252CdeviceType%253D%253Ddesktop&trigger=archivephp. Response was '{"result":"error","message":"Invalid 'log_visit.config_device_type' segment value tablet,deviceType==desktop"}'
ERROR CoreAdminHome[2020-12-22 17:06:30 UTC] [6f5e3] 6 total errors during this script execution, please investigate and try and fix these errors.
ERROR API[2020-12-22 17:06:30 UTC] [6f5e3] Uncaught exception in API: /www/htdocs/w01xxxxx/web/core/CronArchive.php(605): 6 total errors during this script execution, please investigate and try and fix these errors.
  6 total errors during this script execution, please investigate and try and fix these errors.
Error: 6 total errors during this script execution, please investigate and try and fix these errors.

Usually this is being run as a web cronjob every 30 minutes.
Running the System check seems fine to me (except a few suggestions for data integrity which should not be an issue here):

# Mandatory checks

## PHP-Version >= 7.2.5: ✔ 7.4.10

## PDO Erweiterung: ✔ 

## PDO\MYSQL Erweiterung: ✔ 

## MYSQLI Erweiterung: ✔ 
## Weitere erforderliche Anforderungen: ✔ zlib ✔ SPL ✔ iconv ✔ json ✔ mbstring ✔ Reflection
## Erforderliche Funktionen: ✔ debug_backtrace ✔ eval ✔ hash ✔ gzcompress ✔ gzuncompress ✔ pack
## Benötigte PHP Konfiguration (php.ini): ✔ session.auto_start = 0 ✔ max_execution_time = 0 OR >= 30
## Verzeichnisse mit Schreibzugriff: ✔ $DOC_ROOT/tmp ✔ $DOC_ROOT/tmp/assets ✔ $DOC_ROOT/tmp/cache ✔ $DOC_ROOT/tmp/climulti ✔ $DOC_ROOT/tmp/latest ✔ $DOC_ROOT/tmp/logs ✔ $DOC_ROOT/tmp/sessions ✔ $DOC_ROOT/tmp/tcpdf ✔ $DOC_ROOT/tmp/templates_c

# Optional checks

## Dateiintegrität: ⚠ Warning: Der Datei-Integritätscheck ist fehlgeschlagen und hat ein paar Fehler gemeldet. Du solltest die Probleme beheben und diese Seite neuladen, bis sie keine Fehler mehr anzeigt.<br/><br/><pre style="overflow-x: scroll;max-width: 600px;">Es wurden unerwartete Dateien in Ihrer Matomo-Installation gefunden.<br/>--> Bitte löschen Sie diese Dateien, um Fehler zu vermeiden. <--<br/><br/>Zu löschende Datei: js/piwik-min.js<br/>Zu löschende Datei: js/piwik.js.orig<br/>Zu löschende Datei: wartung/backup.php<br/>Zu löschende Datei: usage/.htpasswd<br/><br/><br/>Um alle diese Dateien auf einmal zu löschen, kannst du folgenden Befehl ausführen:<br/>rm "$DOC_ROOT/js/piwik-min.js" "$DOC_ROOT/js/piwik.js.orig" "$DOC_ROOT/wartung/backup.php" "$DOC_ROOT/usage/.htpasswd"<br/><br/></pre>

## Tracker-Status: ✔ 
## Speicherlimit: ✔ 512M
## Zeitzone: ✔ 
## Öffnen einer URL: ✔ curl
## PageSpeed deaktiviert: ✔ 
## GD > 2.x + Freetype (graphics): ✔ 
## Andere Erweiterungen: ✔ json ✔ libxml ✔ dom ✔ SimpleXML
## Andere Funktionen: ✔ shell_exec ✔ set_time_limit ✔ mail ✔ parse_ini_file ✔ glob ✔ gzopen ✔ md5_file
## Dateisystem: ✔ 

## Cron einrichten - Prozesse via CLI steuern: nicht unterstützt (optional)
## Letzter erfolgreicher Abschluss der Archivierung: ✔ Der Archivierungsprozess wurde vor 07:35:58 erfolgreich abgeschlossen.
## Datenbankfähigkeiten: ✔ UTF8mb4 charset ✔ LOAD DATA INFILE ✔ CREATE TEMPORARY TABLES ✔ Changing transaction isolation level
## Maximale Packetgröße: ✔ 
## Erzwungene SSL Verbindung: ✔ 
## Standorterkennung: ✔ 
## Update über HTTPS: ✔ 
## Schreibbarer JavaScript-Tracker ("/matomo.js"): ✔ 

# Informational results

## Matomo Version: 4.1.0
## Matomo Update History: 3.14.1,
## Matomo Install Version: 3.8.0
## PHP_OS: Linux
## PHP_BINARY: /usr/sbin/php74-fpm
## PHP SAPI: fpm-fcgi
## Timezone Version: 0.system
## PHP Timezone: UTC
## PHP Time: 1608660441
## PHP Datetime: 2020-12-22 18:07:21
## PHP INI max_execution_time: 30
## PHP INI post_max_size: 200M
## PHP INI max_input_vars: 3000
## PHP INI zlib.output_compression: 
## Curl Version: 7.68.0, OpenSSL/1.1.1f
## Suhosin Installed: 0

## DB Prefix: matomo_
## DB Charset: utf8mb4
## DB Adapter: PDO\MYSQL
## MySQL Version: 10.5.6-MariaDB-1:10.5.6+maria~focal-log
## Num Tables: 156
## Browser Segment Archiving Enabled: 0
## Development Mode Enabled: 0
## Internet Enabled: 1
## Multi Server Environment: 0
## Custom User Path: 0
## Custom Include Path: 0

## Plugins Activated: API, Actions, Annotations, BulkTracking, Contents, CoreAdminHome, CoreConsole, CoreHome, CorePluginsAdmin, CoreUpdater, CoreVisualizations, CustomDimensions, CustomJsTracker, Dashboard, DevicePlugins, DevicesDetection, Diagnostics, Ecommerce, Events, Feedback, GeoIp2, Goals, Heartbeat, ImageGraph, Insights, Installation, Intl, IntranetMeasurable, LanguagesManager, Live, Login, Marketplace, MobileMessaging, Monolog, Morpheus, MultiSites, Overlay, PagePerformance, PrivacyManager, ProfessionalServices, Provider, Proxy, Referrers, Resolution, RssWidget, SEO, ScheduledReports, SegmentEditor, SitesManager, Tour, Transitions, TwoFactorAuth, UserCountry, UserCountryMap, UserId, UserLanguage, UsersManager, VisitFrequency, VisitTime, VisitorInterest, VisitsSummary, WebsiteMeasurable, Widgetize

## Plugins Deactivated: CustomVariables, DBStats, MobileAppMeasurable, TagManager

## Plugins Invalid: 

## Server Info: Apache

## Had visits in last 1 day: 1
## Had visits in last 3 days: 1
## Had visits in last 5 days: 1
## Archive Time Last Started: 1608660145
## Archive Time Last Finished: 1608633080
## Num invalidations: 1 queued, 0 in progress
## User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36
## Browser Language: de-de,de,en-us,en
## Anonymize Referrer: 
## Do Not Track enabled: 0

Any help to possibly get rid of these six errors with the response

Response was ‘{“result”:“error”,“message”:“Invalid ‘log_visit.config_device_type’ segment value tablet,deviceType==desktop”}’

is greatly appreciated.

Thank you for your help!

Hi @domdom, sorry you’re experiencing this issue. I was able to reproduce this and put a fix up here: https://github.com/matomo-org/matomo/pull/17004 . Would you be able to apply the changes and see if it fixes the problem for you?

Hi @diosmosis, I wasn’t sure if these messages relate to some misconfiguration on my side or if there really might be something going on under the hood which I could not fix myself. So thank you very much for looking into this and also for providing a solution at such a short time!

Your update to the two files worked perfectly and now the overall archive process runs smooth again!

Seems to me that this would be a good fix in the final branch for the next release.

Thanks again and merry christmas!