Import log analytics - No datas shown

toto · February 25, 2020, 10:44am

Some days ago, no problem to import datas and see it.
But now when i do import, it seems to be good, but no datas to see.
I found in the forum a similar problem : Problem with Log Analytics - It seems to import something but no data is shown
The BulkTracking plugin is enabled for me.
Matomo 3.13.2

Here is the result of the import

Logs import summary

1727 requests imported successfully
0 requests were downloads
169 requests ignored:
    0 HTTP errors
    0 HTTP redirects
    4 invalid log lines
    0 filtered log lines
    0 requests did not match any known site
    0 requests did not match any --hostname
    0 requests done by bots, search engines...
    165 requests to static resources (css, js, images, ico, ttf...)
    0 requests to file downloads did not match any --download-extensions

Website import summary

1727 requests imported to 1 sites
    1 sites already existed
    0 sites were created:

0 distinct hostnames did not match any existing site:

Performance summary

Total time: 15 seconds
Requests imported per second: 114.53 requests per second

Processing your log data

In order for your logs to be processed by Matomo, you may need to run the following command:
 ./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url='http://mysite/matomo/'

Here is the result of the archive

INFO [2020-02-25 11:53:43] 6768  ---------------------------
INFO [2020-02-25 11:53:43] 6768  INIT
INFO [2020-02-25 11:53:43] 6768  Running Matomo 3.13.2 as Super User
INFO [2020-02-25 11:53:43] 6768  ---------------------------
INFO [2020-02-25 11:53:43] 6768  NOTES
INFO [2020-02-25 11:53:43] 6768  - Async process archiving not supported, using curl requests.
INFO [2020-02-25 11:53:43] 6768  - Reports for today will be processed at most every 900 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-02-25 11:53:43] 6768  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2020-02-25 11:53:43] 6768  - Archiving was last executed without error 37s ago
INFO [2020-02-25 11:53:43] 6768  - Will process all 4 websites
INFO [2020-02-25 11:53:43] 6768  ---------------------------
INFO [2020-02-25 11:53:43] 6768  START
INFO [2020-02-25 11:53:43] 6768  Starting Matomo reports archiving...
INFO [2020-02-25 11:53:44] 6768  Will pre-process for website id = 1, period = day, date = last1000
INFO [2020-02-25 11:53:44] 6768  - pre-processing all visits
INFO [2020-02-25 11:53:50] 6768  Archived website id = 1, period = day, 0 segments, 2200 visits in last 1000 days, 0 visits today, Time elapsed: 6.746s
INFO [2020-02-25 11:53:50] 6768  Will pre-process for website id = 1, period = week, date = last520
INFO [2020-02-25 11:53:50] 6768  - pre-processing all visits
INFO [2020-02-25 11:53:53] 6768  Archived website id = 1, period = week, 0 segments, 0 visits in last 1000 weeks, 0 visits this week, Time elapsed: 3.309s
INFO [2020-02-25 11:53:53] 6768  Will pre-process for website id = 1, period = month, date = last120
INFO [2020-02-25 11:53:53] 6768  - pre-processing all visits
INFO [2020-02-25 11:53:55] 6768  Archived website id = 1, period = month, 0 segments, 0 visits in last 1000 months, 0 visits this month, Time elapsed: 1.167s
INFO [2020-02-25 11:53:55] 6768  Will pre-process for website id = 1, period = year, date = last10
INFO [2020-02-25 11:53:55] 6768  - pre-processing all visits
INFO [2020-02-25 11:53:55] 6768  Archived website id = 1, period = year, 0 segments, 0 visits in last 1000 years, 0 visits this year, Time elapsed: 0.479s
INFO [2020-02-25 11:53:55] 6768  Archived website id = 1, 4 API requests, Time elapsed: 11.721s [1/4 done]

...


INFO [2020-02-25 11:54:28] 6768  ---------------------------
INFO [2020-02-25 11:54:28] 6768  SUMMARY
INFO [2020-02-25 11:54:28] 6768  Total visits for today across archived websites: 0
INFO [2020-02-25 11:54:28] 6768  Archived today's reports for 4 websites
INFO [2020-02-25 11:54:28] 6768  Archived week/month/year for 4 websites
INFO [2020-02-25 11:54:28] 6768  Skipped 0 websites
INFO [2020-02-25 11:54:28] 6768  - 0 skipped because no new visit since the last script execution
INFO [2020-02-25 11:54:28] 6768  - 0 skipped because existing daily reports are less than 900 seconds old
INFO [2020-02-25 11:54:28] 6768  - 0 skipped because existing week/month/year periods reports are less than 3600 seconds old
INFO [2020-02-25 11:54:28] 6768  Total API requests: 16
INFO [2020-02-25 11:54:28] 6768  done: 4/4 100%, 0 vtoday, 4 wtoday, 4 wperiods, 16 req, 44626 ms, no error
INFO [2020-02-25 11:54:28] 6768  Time elapsed: 44.626s
INFO [2020-02-25 11:54:28] 6768  ---------------------------
INFO [2020-02-25 11:54:28] 6768  SCHEDULED TASKS
INFO [2020-02-25 11:54:28] 6768  Starting Scheduled tasks...
INFO [2020-02-25 11:54:28] 6768  done
INFO [2020-02-25 11:54:28] 6768  ---------------------------

kwisarts · March 2, 2020, 8:22pm

I’m having quite a similar situation, also with 3.13.2. I imported logs for the past 7 days (files are rotated weekly) but the archival process reports 0 visits and 0 visits is what I’m seeing on the front-end too for the imported date-range:

root@org-matomo:/var/www/matomo/core/Tracker# sudo -u www-data /usr/bin/php /var/www/matomo/console core:invalidate-report-data --dates=2020-02-23,2020-03-01 --sites=4
Invalidating day periods in 2020-02-23,2020-03-01 [segment = ]...
Invalidating week periods in 2020-02-23,2020-03-01 [segment = ]...
Invalidating month periods in 2020-02-23,2020-03-01 [segment = ]...
Invalidating year periods in 2020-02-23,2020-03-01 [segment = ]...

root@org-matomo:/var/www/matomo/core/Tracker# sudo -u www-data python /var/www/matomo/misc/log-analytics/import_logs.py --url=https://stats.otherdomain.com --recorders=2 --idsite=4 --enable-http-errors --tracker-endpoint=/piwik.php?queuedtracking=0 /home/matomo/logs/*domain.com.access.log-20200301
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log /home/matomo/logs/domain.com.access.log-20200301...
15033 lines parsed, 809 lines recorded, 808 records/sec (avg), 809 records/sec (current)
20024 lines parsed, 2003 lines recorded, 1000 records/sec (avg), 1194 records/sec (current)
28779 lines parsed, 3206 lines recorded, 1067 records/sec (avg), 1203 records/sec (current)
35204 lines parsed, 4403 lines recorded, 1099 records/sec (avg), 1197 records/sec (current)
43187 lines parsed, 6007 lines recorded, 1199 records/sec (avg), 1604 records/sec (current)
49485 lines parsed, 7601 lines recorded, 1265 records/sec (avg), 1594 records/sec (current)
58124 lines parsed, 8807 lines recorded, 1255 records/sec (avg), 1206 records/sec (current)
64347 lines parsed, 10410 lines recorded, 1298 records/sec (avg), 1603 records/sec (current)
72309 lines parsed, 12002 lines recorded, 1330 records/sec (avg), 1592 records/sec (current)
79241 lines parsed, 13602 lines recorded, 1357 records/sec (avg), 1600 records/sec (current)
Parsing log /home/matomo/logs/preview.domain.com.access.log-20200301...
Parsing log /home/matomo/logs/ssl-domain.com.access.log-20200301...
Parsing log /home/matomo/logs/www.domain.com.access.log-20200301...                              
84553 lines parsed, 15272 lines recorded, 1385 records/sec (avg), 1670 records/sec (current) 
90762 lines parsed, 17629 lines recorded, 1466 records/sec (avg), 2357 records/sec (current)
96834 lines parsed, 20238 lines recorded, 1553 records/sec (avg), 2609 records/sec (current)
100784 lines parsed, 22418 lines recorded, 1598 records/sec (avg), 2180 records/sec (current)
106469 lines parsed, 25201 lines recorded, 1676 records/sec (avg), 2783 records/sec (current)
111021 lines parsed, 27192 lines recorded, 1696 records/sec (avg), 1991 records/sec (current)
116554 lines parsed, 29669 lines recorded, 1742 records/sec (avg), 2477 records/sec (current)
121650 lines parsed, 32072 lines recorded, 1778 records/sec (avg), 2403 records/sec (current)
126608 lines parsed, 34476 lines recorded, 1811 records/sec (avg), 2404 records/sec (current)
132269 lines parsed, 36809 lines recorded, 1836 records/sec (avg), 2333 records/sec (current)
137218 lines parsed, 38706 lines recorded, 1839 records/sec (avg), 1897 records/sec (current)
142114 lines parsed, 41022 lines recorded, 1860 records/sec (avg), 2316 records/sec (current)
146085 lines parsed, 43138 lines recorded, 1871 records/sec (avg), 2116 records/sec (current)
151147 lines parsed, 45800 lines recorded, 1904 records/sec (avg), 2662 records/sec (current)
156836 lines parsed, 48193 lines recorded, 1924 records/sec (avg), 2393 records/sec (current)
161288 lines parsed, 50661 lines recorded, 1944 records/sec (avg), 2468 records/sec (current)
166352 lines parsed, 52843 lines recorded, 1953 records/sec (avg), 2182 records/sec (current)
171302 lines parsed, 55289 lines recorded, 1971 records/sec (avg), 2446 records/sec (current)
175361 lines parsed, 57294 lines recorded, 1972 records/sec (avg), 2005 records/sec (current)
180212 lines parsed, 59822 lines recorded, 1990 records/sec (avg), 2528 records/sec (current)
184738 lines parsed, 62086 lines recorded, 1999 records/sec (avg), 2264 records/sec (current)
190132 lines parsed, 64387 lines recorded, 2008 records/sec (avg), 2301 records/sec (current)
195668 lines parsed, 66925 lines recorded, 2024 records/sec (avg), 2538 records/sec (current)
201198 lines parsed, 69083 lines recorded, 2028 records/sec (avg), 2158 records/sec (current)
206348 lines parsed, 71272 lines recorded, 2032 records/sec (avg), 2189 records/sec (current)
211416 lines parsed, 73827 lines recorded, 2047 records/sec (avg), 2555 records/sec (current)
216289 lines parsed, 76337 lines recorded, 2059 records/sec (avg), 2510 records/sec (current)
221464 lines parsed, 78541 lines recorded, 2063 records/sec (avg), 2204 records/sec (current)
225984 lines parsed, 80644 lines recorded, 2064 records/sec (avg), 2103 records/sec (current)
230214 lines parsed, 83358 lines recorded, 2080 records/sec (avg), 2714 records/sec (current)
235894 lines parsed, 85842 lines recorded, 2090 records/sec (avg), 2484 records/sec (current)
240161 lines parsed, 87783 lines recorded, 2086 records/sec (avg), 1941 records/sec (current)
245388 lines parsed, 89902 lines recorded, 2087 records/sec (avg), 2119 records/sec (current)
248823 lines parsed, 92069 lines recorded, 2088 records/sec (avg), 2167 records/sec (current)
254391 lines parsed, 94472 lines recorded, 2095 records/sec (avg), 2403 records/sec (current)
260393 lines parsed, 97027 lines recorded, 2105 records/sec (avg), 2555 records/sec (current)
264656 lines parsed, 99242 lines recorded, 2107 records/sec (avg), 2215 records/sec (current)
270117 lines parsed, 101529 lines recorded, 2111 records/sec (avg), 2287 records/sec (current)
275638 lines parsed, 104072 lines recorded, 2119 records/sec (avg), 2543 records/sec (current)
277721 lines parsed, 106082 lines recorded, 2117 records/sec (avg), 2010 records/sec (current)

Logs import summary
-------------------

    106082 requests imported successfully
    2217 requests were downloads
    171639 requests ignored:
        0 HTTP errors
        15735 HTTP redirects
        0 invalid log lines
        0 filtered log lines
        0 requests did not match any known site
        0 requests did not match any --hostname
        11782 requests done by bots, search engines...
        144122 requests to static resources (css, js, images, ico, ttf...)
        0 requests to file downloads did not match any --download-extensions

Website import summary
----------------------

    106082 requests imported to 1 sites
        1 sites already existed
        0 sites were created:

    0 distinct hostnames did not match any existing site:



Performance summary
-------------------

    Total time: 50 seconds
    Requests imported per second: 2097.76 requests per second

Processing your log data
------------------------

    In order for your logs to be processed by Matomo, you may need to run the following command:
     ./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url='https://stats.otherdomain.com'

root@org-matomo:/var/www/matomo/core/Tracker# sudo -u www-data /usr/bin/php /var/www/matomo/console core:archive --force-idsites=4 --force-date-last-n=8 --force-all-periods=315576000 --url='https://stats.otherdomain.com'
INFO [2020-03-02 20:12:27] 31157  ---------------------------
INFO [2020-03-02 20:12:27] 31157  INIT
INFO [2020-03-02 20:12:27] 31157  Running Matomo 3.13.3 as Super User
INFO [2020-03-02 20:12:27] 31157  ---------------------------
INFO [2020-03-02 20:12:27] 31157  NOTES
INFO [2020-03-02 20:12:27] 31157  - Async process archiving supported, using CliMulti.
INFO [2020-03-02 20:12:27] 31157  - Reports for today will be processed at most every 150 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-03-02 20:12:27] 31157  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2020-03-02 20:12:27] 31157  - Archiving was last executed without error 1 hours 5 min ago
INFO [2020-03-02 20:12:27] 31157  - Will process 1 websites (--force-idsites)
INFO [2020-03-02 20:12:27] 31157  ---------------------------
INFO [2020-03-02 20:12:27] 31157  START
INFO [2020-03-02 20:12:27] 31157  Starting Matomo reports archiving...
INFO [2020-03-02 20:12:27] 31157  Will pre-process for website id = 4, period = day, date = last8
INFO [2020-03-02 20:12:27] 31157  - pre-processing all visits
INFO [2020-03-02 20:12:29] 31157  Archived website id = 4, period = day, 0 segments, 0 visits in last 8 days, 0 visits today, Time elapsed: 1.562s
INFO [2020-03-02 20:12:29] 31157  Will pre-process for website id = 4, period = week, date = last3
INFO [2020-03-02 20:12:29] 31157  - pre-processing all visits
INFO [2020-03-02 20:12:31] 31157  Archived website id = 4, period = week, 0 segments, 0 visits in last 8 weeks, 0 visits this week, Time elapsed: 2.744s
INFO [2020-03-02 20:12:32] 31157  Will pre-process for website id = 4, period = month, date = last2
INFO [2020-03-02 20:12:32] 31157  - pre-processing all visits
INFO [2020-03-02 20:12:34] 31157  Archived website id = 4, period = month, 0 segments, 0 visits in last 8 months, 0 visits this month, Time elapsed: 2.797s
INFO [2020-03-02 20:12:34] 31157  Will pre-process for website id = 4, period = year, date = last8
INFO [2020-03-02 20:12:34] 31157  - pre-processing all visits
INFO [2020-03-02 20:12:36] 31157  Archived website id = 4, period = year, 0 segments, 1874600 visits in last 8 years, 102177 visits this year, Time elapsed: 1.469s
INFO [2020-03-02 20:12:36] 31157  Will pre-process for website id = 4, period = range, date = previous30
INFO [2020-03-02 20:12:36] 31157  - pre-processing all visits
INFO [2020-03-02 20:12:37] 31157  Archived website id = 4, period = range, 0 segments, 30057 visits in previous 30 ranges, 30057 visits this range, Time elapsed: 1.585s
INFO [2020-03-02 20:12:37] 31157  Archived website id = 4, 5 API requests, Time elapsed: 10.178s [1/1 done]
INFO [2020-03-02 20:12:37] 31157  Done archiving!
INFO [2020-03-02 20:12:37] 31157  ---------------------------
INFO [2020-03-02 20:12:37] 31157  SUMMARY
INFO [2020-03-02 20:12:37] 31157  Total visits for today across archived websites: 0
INFO [2020-03-02 20:12:37] 31157  Archived today's reports for 1 websites
INFO [2020-03-02 20:12:37] 31157  Archived week/month/year for 1 websites
INFO [2020-03-02 20:12:37] 31157  Skipped 17 websites
INFO [2020-03-02 20:12:37] 31157  - 0 skipped because no new visit since the last script execution
INFO [2020-03-02 20:12:37] 31157  - 0 skipped because existing daily reports are less than 150 seconds old
INFO [2020-03-02 20:12:37] 31157  - 0 skipped because existing week/month/year periods reports are less than 3600 seconds old
INFO [2020-03-02 20:12:37] 31157  Total API requests: 5
INFO [2020-03-02 20:12:37] 31157  done: 1/1 100%, 0 vtoday, 1 wtoday, 1 wperiods, 5 req, 10217 ms, no error
INFO [2020-03-02 20:12:37] 31157  Time elapsed: 10.218s
INFO [2020-03-02 20:12:37] 31157  ---------------------------
INFO [2020-03-02 20:12:37] 31157  SCHEDULED TASKS
INFO [2020-03-02 20:12:37] 31157  Starting Scheduled tasks... 
INFO [2020-03-02 20:12:37] 31157  Scheduler: executing task Piwik\Plugins\CustomPiwikJs\Tasks.updateTracker...
INFO [2020-03-02 20:12:37] 31157  Scheduler: finished. Time elapsed: 0.001s
INFO [2020-03-02 20:12:37] 31157  Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.deleteLogData...
INFO [2020-03-02 20:12:37] 31157  Scheduler: finished. Time elapsed: 0.008s
INFO [2020-03-02 20:12:37] 31157  Scheduler: executing task Piwik\Plugins\PrivacyManager\Tasks.anonymizePastData...
INFO [2020-03-02 20:12:37] 31157  Scheduler: finished. Time elapsed: 0.000s
INFO [2020-03-02 20:12:37] 31157  Scheduler: executing task Piwik\Plugins\QueuedTracking\Tasks.notifyQueueSize...
INFO [2020-03-02 20:12:37] 31157  Scheduler: finished. Time elapsed: 0.004s
INFO [2020-03-02 20:12:37] 31157  done
INFO [2020-03-02 20:12:37] 31157  ---------------------------

kwisarts · March 3, 2020, 11:22am

I reiterated the entire this circle after a nights sleep – tl;dr – to no avail.

Invalidated (this time for site with IDs 4 and 6), re-imported the logs, then re-ran core:archive (once with the QueuedTracking plug-in enabled, once disabled). With an n > 8, it tells me there are visits of course, but the results for March are still the same though:

For idsite 4, having `queuedTracking` enabled:

INFO [2020-03-03 11:04:08] 5394  NOTES
INFO [2020-03-03 11:04:08] 5394  - Async process archiving supported, using CliMulti.
INFO [2020-03-03 11:04:08] 5394  - Reports for today will be processed at most every 150 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-03-03 11:04:08] 5394  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2020-03-03 11:04:08] 5394  - Archiving was last executed without error 3 hours 59 min ago
INFO [2020-03-03 11:04:08] 5394  - Will process all 18 websites
INFO [2020-03-03 11:04:08] 5394  Will ignore websites and help finish a previous started queue instead. IDs: 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 21
INFO [2020-03-03 11:04:08] 5394 ---------------------------
INFO [2020-03-03 11:04:08] 5394  START
INFO [2020-03-03 11:04:08] 5394  Starting Matomo reports archiving...
INFO [2020-03-03 11:04:08] 5394  Will pre-process for website id = 4, period = day, date = last1000
INFO [2020-03-03 11:04:08] 5394  - pre-processing all visits
INFO [2020-03-03 11:04:27] 5394  Archived website id = 4, period = day, 0 segments, 1874600 visits in last 1000 days, 0 visits today, Time elapsed: 18.504s
INFO [2020-03-03 11:04:27] 5394  Will pre-process for website id = 4, period = week, date = last520

For idsite 6, after deactivating `queuedTracking`:

INFO [2020-03-03 11:12:39] 17232  ---------------------------
INFO [2020-03-03 11:12:39] 17232  INIT
INFO [2020-03-03 11:12:39] 17232  Running Matomo 3.13.3 as Super User
INFO [2020-03-03 11:12:39] 17232  ---------------------------
INFO [2020-03-03 11:12:39] 17232  NOTES
INFO [2020-03-03 11:12:39] 17232  - Async process archiving supported, using CliMulti.
INFO [2020-03-03 11:12:39] 17232  - Reports for today will be processed at most every 150 seconds. You can change this value in Matomo UI > Settings > General Settings.
INFO [2020-03-03 11:12:39] 17232  - Reports for the current week/month/year will be requested at most every 3600 seconds.
INFO [2020-03-03 11:12:39] 17232  - Archiving was last executed without error 3 min 15s ago
INFO [2020-03-03 11:12:39] 17232  - Will process 1 other websites because some old data reports have been invalidated (eg. using the Log Import script or the InvalidateReports plugin) , IDs: 6
INFO [2020-03-03 11:12:39] 17232  - Will process all 18 websites
INFO [2020-03-03 11:12:39] 17232  ---------------------------
INFO [2020-03-03 11:12:39] 17232  START
INFO [2020-03-03 11:12:39] 17232  Starting Matomo reports archiving...
[…]
INFO [2020-03-03 11:13:54] 17232  Will pre-process for website id = 6, period = day, date = last1000
INFO [2020-03-03 11:14:12] 17232  Archived website id = 6, period = day, 0 segments, 817154 visits in last 1000 days, 0 visits today, Time elapsed: 18.771s
[…]

The same, for 9 days into the past only:

INFO [2020-03-03 11:23:54] 29518  START                                                                                                                                                                                                                                                                                        
INFO [2020-03-03 11:23:54] 29518  Starting Matomo reports archiving...                                                                                                                                                                                                                                                         
INFO [2020-03-03 11:23:54] 29518  Will pre-process for website id = 4, period = day, date = last9                                                                                                                                                                                                                              
INFO [2020-03-03 11:23:54] 29518  - pre-processing all visits                                                                                                                                                                                                                                                                  
INFO [2020-03-03 11:23:56] 29518  Archived website id = 4, period = day, 0 segments, 0 visits in last 9 days, 0 visits today, Time elapsed: 1.541s

Note how it again says “0” visits.

And the database records:

MariaDB [piwik]> select name, idsite, ts_archived, date1, date2, value from archive_blob_2020_03 where idsite IN (4,6);
Empty set (0.001 sec)

toto · March 5, 2020, 3:31pm

I did a fresh install of Matomo.
Same problem.

So, i’ve tried an other thing.
I’ve already some imported datas before 2020 / 02 / 20.
It’s ok with this datas.

I imported datas for the 2020 / 03 / 04.
It’s not OK : same problem as described before.

So, i imported datas for 2020 / 02 / 21 - 2020 / 02 / 22 - 2020 / 02 / 23 - 2020 / 02 / 24 - 2020 / 02 / 25 - 2020 / 02 / 26 - 2020 / 02 / 27 - 2020 / 02 / 28 - 2020 / 02 / 29 - 2020 / 01 03, one day per one day.
I did the archive.
Datas for 2020 / 02 / 21 to 2020 / 01 / 03 are OK and displayed in the graphs.

I imported datas for 2020 / 03 / 04, but not for 2020 / 03 / 02 and not 2020 / 03 / 03.
I did the archive.
Datas for 2020 / 03 / 04 are OK and displayed in the graphs.

I’m waiting for another few days to see if the same thing happens.

kwisarts · March 8, 2020, 3:14pm

@toto So you’re saying if you only import data for March, it displays correctly, but if you import it into a DB that already contains older data, it does not?

In my case I just checked the log_visit table, detecting that, despite the import_logs.py script claiming that lines were recorded, none actually are. E.g.:

MariaDB [org_piwik]> select count(distinct log_visit.idvisitor) from log_visit where DATE(log_visit.visit_last_action_time) > '2020-02-24' and log_visit.idsite IN (4,6);
+-------------------------------------+
| count(distinct log_visit.idvisitor) |
+-------------------------------------+
|                                   0 |
+-------------------------------------+
1 row in set (2.116 sec)

I’m planning on enabling further debugging as per Tracking HTTP API: API Reference - Matomo Analytics (formerly Piwik Analytics) - Developer Docs - v3 to hopefully figure out why these requests are not tracked/recorded.

kwisarts · March 8, 2020, 3:56pm

Ok, I tracked down the problem and, in a nutshell, it boils down to Seeking help to understand Tracker configuration for imports using import_logs.py again

In Core\Tracker\Request::getIpString(), the IP extracted from the logs is being replaced by the IP from the headers if token_auth is not set or invalid:

    /**
     * @return mixed|string
     * @throws Exception
     */
    public function getIpString()
    {
        $cip = $this->getParam('cip');

        Common::printDebug(sprintf('CIP is %s', $cip));

        if (empty($cip)) {
            return IP::getIpFromHeader();
        }

        if (!$this->isAuthenticated()) {
            Common::printDebug("WARN: Tracker API 'cip' was used with invalid token_auth");
            return IP::getIpFromHeader();
        }

        return $cip;
    }

I think I might turn this into a bug report on github, since at the very least I think it should respect the setting for tracking_requests_require_authentication_when_custom_timestamp_newer_than.

kwisarts · March 8, 2020, 4:02pm

Someone already stumbled over this and then the README was modified.

The reason I didn’t see this is that the FAQ/Howto does not make any mention of it.