Edit: Same issue as described here: https://github.com/matomo-org/matomo-log-analytics/issues/369
I am trying to import logs from an nginx server and failing miserably.
Let’s just take a single log-file/day. The output from the
python3 misc/log-analytics/import_logs.py --recorders 4 --idsite 25 -dddd --debug-tracker --url=https://my.domain.tld/ --token-auth=********** --show-progress --enable-bots ~/matomo/20250711-access.log
2025-07-20 19:25:12,248: [DEBUG] Accepted hostnames: all
2025-07-20 19:25:12,248: [DEBUG] Matomo Tracker API URL is: https://my.domain.tld/
2025-07-20 19:25:12,248: [DEBUG] Matomo Analytics API URL is: https://my.domain.tld/
2025-07-20 19:25:12,248: [DEBUG] Authentication token token_auth is: **************
2025-07-20 19:25:12,248: [DEBUG] Resolver: static
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
2025-07-20 19:25:12,420: [DEBUG] Launched recorder
2025-07-20 19:25:12,420: [DEBUG] Launched recorder
2025-07-20 19:25:12,420: [DEBUG] Launched recorder
2025-07-20 19:25:12,420: [DEBUG] Launched recorder
Parsing log /root/matomo/20250711-access.log...
2025-07-20 19:25:12,421: [DEBUG] Detecting the log format
2025-07-20 19:25:12,421: [DEBUG] Check format common
2025-07-20 19:25:12,421: [DEBUG] Format common matches
2025-07-20 19:25:12,421: [DEBUG] Format match contains 8 groups
2025-07-20 19:25:12,421: [DEBUG] Check format common_vhost
2025-07-20 19:25:12,421: [DEBUG] Format common_vhost does not match
2025-07-20 19:25:12,421: [DEBUG] Check format ncsa_extended
2025-07-20 19:25:12,421: [DEBUG] Format ncsa_extended matches
2025-07-20 19:25:12,421: [DEBUG] Format match contains 10 groups
2025-07-20 19:25:12,421: [DEBUG] Check format common_complete
2025-07-20 19:25:12,421: [DEBUG] Format common_complete does not match
2025-07-20 19:25:12,421: [DEBUG] Check format w3c_extended
2025-07-20 19:25:12,421: [DEBUG] Format w3c_extended does not match
2025-07-20 19:25:12,421: [DEBUG] Check format amazon_cloudfront
2025-07-20 19:25:12,421: [DEBUG] Format amazon_cloudfront does not match
2025-07-20 19:25:12,421: [DEBUG] Check format incapsula_w3c
2025-07-20 19:25:12,421: [DEBUG] Format incapsula_w3c does not match
2025-07-20 19:25:12,421: [DEBUG] Check format iis
2025-07-20 19:25:12,421: [DEBUG] Format iis does not match
2025-07-20 19:25:12,421: [DEBUG] Check format shoutcast
2025-07-20 19:25:12,421: [DEBUG] Format shoutcast does not match
2025-07-20 19:25:12,421: [DEBUG] Check format s3
2025-07-20 19:25:12,421: [DEBUG] Format s3 does not match
2025-07-20 19:25:12,421: [DEBUG] Check format icecast2
2025-07-20 19:25:12,421: [DEBUG] Format icecast2 does not match
2025-07-20 19:25:12,421: [DEBUG] Check format elb
2025-07-20 19:25:12,421: [DEBUG] Format elb does not match
2025-07-20 19:25:12,421: [DEBUG] Check format traefik_json
2025-07-20 19:25:12,421: [DEBUG] Format traefik_json does not match
2025-07-20 19:25:12,421: [DEBUG] Check format nginx_json
2025-07-20 19:25:12,421: [DEBUG] Format nginx_json does not match
2025-07-20 19:25:12,421: [DEBUG] Check format ovh
2025-07-20 19:25:12,421: [DEBUG] Check format haproxy
2025-07-20 19:25:12,421: [DEBUG] Format haproxy does not match
2025-07-20 19:25:12,421: [DEBUG] Check format gandi
2025-07-20 19:25:12,421: [DEBUG] Format gandi does not match
2025-07-20 19:25:12,421: [DEBUG] Format ncsa_extended is the best match
2025-07-20 19:25:12,448: [DEBUG] Invalid line detected (line did not match): 206.189.149.175 - - [11/Jul/2025:05:12:08 +0200] "" 400 0 "-" "-"
2025-07-20 19:25:12,448: [DEBUG] Invalid line detected (line did not match): 206.189.149.175 - - [11/Jul/2025:05:12:17 +0200] "" 400 0 "-" "-"
2025-07-20 19:25:12,449: [DEBUG] Invalid line detected (line did not match): 206.189.149.175 - - [11/Jul/2025:05:12:31 +0200] "" 400 0 "-" "-"
2025-07-20 19:25:12,449: [DEBUG] Invalid line detected (line did not match): 206.189.149.175 - - [11/Jul/2025:05:13:16 +0200] "" 400 0 "-" "-"
2025-07-20 19:25:12,458: [DEBUG] Invalid line detected (line did not match): 173.252.87.6 - - [11/Jul/2025:09:05:55 +0200] "-" 000 0 "-" "-"
2025-07-20 19:25:12,761: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":35}
2025-07-20 19:25:12,761: [INFO] The Matomo tracker identified 35 invalid requests.
2025-07-20 19:25:12,789: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":107}
2025-07-20 19:25:12,790: [INFO] The Matomo tracker identified 107 invalid requests.
2025-07-20 19:25:12,797: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":209}
2025-07-20 19:25:12,797: [INFO] The Matomo tracker identified 209 invalid requests.
2025-07-20 19:25:12,847: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":449}
2025-07-20 19:25:12,847: [INFO] The Matomo tracker identified 449 invalid requests.
2025-07-20 19:25:12,946: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":34}
2025-07-20 19:25:12,946: [INFO] The Matomo tracker identified 34 invalid requests.
2025-07-20 19:25:12,950: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":73}
2025-07-20 19:25:12,950: [INFO] The Matomo tracker identified 73 invalid requests.
2025-07-20 19:25:13,019: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":267}
2025-07-20 19:25:13,019: [INFO] The Matomo tracker identified 267 invalid requests.
2025-07-20 19:25:13,080: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":426}
2025-07-20 19:25:13,080: [INFO] The Matomo tracker identified 426 invalid requests.
2025-07-20 19:25:13,122: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":23}
2025-07-20 19:25:13,123: [INFO] The Matomo tracker identified 23 invalid requests.
2025-07-20 19:25:13,163: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":340}
2025-07-20 19:25:13,163: [INFO] The Matomo tracker identified 340 invalid requests.
2025-07-20 19:25:13,173: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":75}
2025-07-20 19:25:13,173: [INFO] The Matomo tracker identified 75 invalid requests.
2025-07-20 19:25:13,299: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":37}
2025-07-20 19:25:13,299: [INFO] The Matomo tracker identified 37 invalid requests.
2025-07-20 19:25:13,316: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":362}
2025-07-20 19:25:13,317: [INFO] The Matomo tracker identified 362 invalid requests.
2025-07-20 19:25:13,325: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":40}
2025-07-20 19:25:13,326: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":63}
2025-07-20 19:25:13,326: [INFO] The Matomo tracker identified 63 invalid requests.
2025-07-20 19:25:13,326: [INFO] The Matomo tracker identified 40 invalid requests.
4336 lines parsed, 2540 lines recorded, 2538 records/sec (avg), 2540 records/sec (current)
2025-07-20 19:25:13,485: [DEBUG] tracker response:
{"status":"success","tracked":0,"invalid":377}
2025-07-20 19:25:13,485: [INFO] The Matomo tracker identified 377 invalid requests.
Logs import summary
-------------------
2917 requests imported successfully
65 requests were downloads
1419 requests ignored:
131 HTTP errors
373 HTTP redirects
5 invalid log lines
0 filtered log lines
0 requests did not match any known site
0 requests did not match any --hostname
0 requests done by bots, search engines...
910 requests to static resources (css, js, images, ico, ttf...)
0 requests to file downloads did not match any --download-extensions
Website import summary
----------------------
2917 requests imported to 1 sites
1 sites already existed
0 sites were created:
0 distinct hostnames did not match any existing site:
Performance summary
-------------------
Total time: 1 seconds
Requests imported per second: 2034.18 requests per second
So there are two things here. For one, it does say that it imported 2917 requests… but in the tracker debugging section, it says that is actually tracked 0 requests.
Following this up with a core:archive command, I get told that there is no data for the period in question:
DEBUG [17:32:08] 2282967 Found invalidated archive we can skip (no visits): [idinvalidation = 8597, idsite = 25, period = day(2025-07-11 - 2025-07-11), name = done, segment = ]
Full log output:
root@adsns-web-node0:~# sudo -u web157 /usr/bin/php8.3 /var/www/clients/client53/web157/web/console core:archive --url='https://my.domain.tld/' --force-idsites=25 --force-date-range=2025-07-11,2025-07-11 --force-periods=day -vvv
INFO [17:32:08] 2282967 ---------------------------
INFO [17:32:08] 2282967 INIT
INFO [17:32:08] 2282967 Running Matomo 5.0.1 as Super User
INFO [17:32:08] 2282967 ---------------------------
INFO [17:32:08] 2282967 NOTES
INFO [17:32:08] 2282967 - Async process archiving supported, using CliMulti.
INFO [17:32:08] 2282967 - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [17:32:08] 2282967 - Archiving was last executed without error 26 min 8s ago.
INFO [17:32:08] 2282967 - Will process 1 websites (--force-idsites)
INFO [17:32:08] 2282967 - Will process specified sites: 25
INFO [17:32:08] 2282967 - Will only process the following periods: day (--force-periods)
INFO [17:32:08] 2282967 ---------------------------
INFO [17:32:08] 2282967 START
INFO [17:32:08] 2282967 Starting Matomo reports archiving...
INFO [17:32:08] 2282967 1 out of 3 archivers running currently
DEBUG [17:32:08] 2282967 Applying queued rearchiving...
INFO [17:32:08] 2282967 Start processing archives for site 25.
DEBUG [17:32:08] 2282967 Checking for queued invalidations...
INFO [17:32:08] 2282967 Will invalidate archived reports for today in site ID = 25's timezone (2025-07-20 00:00:00).
INFO [17:32:08] 2282967 Will invalidate archived reports for yesterday in site ID = 25's timezone (2025-07-19 00:00:00).
DEBUG [17:32:08] 2282967 Found usable archive for [idSite = 25, period = day 2025-07-19,2025-07-19, segment = , plugin = , report = ], skipping invalidation.
DEBUG [17:32:08] 2282967 Done invalidating
DEBUG [17:32:08] 2282967 Skipping invalidated archive 8483, archive date range (2025-07-20,2025-07-20) is not within --force-date-range: [idinvalidation = 8483, idsite = 25, period = day(2025-07-20 - 2025-07-20), name = done, segment = ]
DEBUG [17:32:08] 2282967 Skipping invalidated archive 8581, archive date range (2025-07-14,2025-07-20) is not within --force-date-range: [idinvalidation = 8581, idsite = 25, period = week(2025-07-14 - 2025-07-20), name = done, segment = ]
DEBUG [17:32:08] 2282967 No usable archive exists (ts_archived of existing = , now = 2025-07-20 17:32:08).
DEBUG [17:32:08] 2282967 Found invalidated archive we can skip (no visits): [idinvalidation = 8597, idsite = 25, period = day(2025-07-11 - 2025-07-11), name = done, segment = ]
DEBUG [17:32:08] 2282967 Skipping invalidated archive 8591, period is not specified in --force-periods: [idinvalidation = 8591, idsite = 25, period = week(2025-07-07 - 2025-07-13), name = done, segment = ]
DEBUG [17:32:08] 2282967 Skipping invalidated archive 8593, period is not specified in --force-periods: [idinvalidation = 8593, idsite = 25, period = month(2025-07-01 - 2025-07-31), name = done, segment = ]
DEBUG [17:32:08] 2282967 Skipping invalidated archive 8595, period is not specified in --force-periods: [idinvalidation = 8595, idsite = 25, period = year(2025-01-01 - 2025-12-31), name = done, segment = ]
DEBUG [17:32:08] 2282967 No next invalidated archive.
INFO [17:32:08] 2282967 Finished archiving for site 25, 0 API requests, Time elapsed: 0.149s [1 / 1 done]
DEBUG [17:32:08] 2282967 No more sites left to archive, stopping.
INFO [17:32:08] 2282967 Done archiving!
INFO [17:32:08] 2282967 ---------------------------
INFO [17:32:08] 2282967 SUMMARY
INFO [17:32:08] 2282967 Processed 0 archives.
INFO [17:32:08] 2282967 Total API requests: 0
INFO [17:32:08] 2282967 done: 0 req, 201 ms, no error
INFO [17:32:08] 2282967 Time elapsed: 0.201s
INFO [17:32:08] 2282967 ---------------------------
INFO [17:32:08] 2282967 SCHEDULED TASKS
DEBUG [17:32:08] 2282967 27 scheduled tasks loaded
INFO [17:32:08] 2282967 Starting Scheduled tasks...
DEBUG [17:32:08] 2282967 Executing tasks with priority 0:
DEBUG [17:32:08] 2282967 Executing tasks with priority 1:
DEBUG [17:32:08] 2282967 Executing tasks with priority 2:
DEBUG [17:32:08] 2282967 Executing tasks with priority 3:
DEBUG [17:32:08] 2282967 Executing tasks with priority 4:
DEBUG [17:32:08] 2282967 Executing tasks with priority 5:
DEBUG [17:32:08] 2282967 Executing tasks with priority 6:
DEBUG [17:32:08] 2282967 Executing tasks with priority 7:
DEBUG [17:32:08] 2282967 Executing tasks with priority 8:
DEBUG [17:32:08] 2282967 Executing tasks with priority 9:
DEBUG [17:32:08] 2282967 Executing tasks with priority 10:
DEBUG [17:32:08] 2282967 Executing tasks with priority 11:
DEBUG [17:32:08] 2282967 Executing tasks with priority 12:
INFO [17:32:08] 2282967 done
INFO [17:32:08] 2282967 ---------------------------
I also tried running the climulti command, with the same outcome:
/usr/bin/php8.3 -q /var/www/…/web/console climulti:request -q --matomo-domain='my.domain.tld' --superuser 'module=API&method=CoreAdminHome.archiveReports&idSite=25&period=day&date=2025-07-11&format=json&trigger=archivephp'
{"idarchives":false,"nb_visits":0}
Can anyone confirmed that the tracker is the problem here and if so, suggest how to further debug this?
Thanks in advance!