I’m trying to understand how the tracking_requests_require_authentication_when_custom_timestamp_newer_than configuration option plays into log-analytics’ import_logs.py.
E.g.
root@org-matomo:/var/www/matomo/core/Tracker# sudo -u www-data python /var/www/matomo/misc/log-analytics/import_logs.py --url=https://domain.com --recorders=2 --idsite=4 --enable-http-errors --tracker-endpoint=/piwik.php?queuedtracking=0 /home/matomo/logs/*domain.com.access.log-20200301
(This might be related to Lots of invalid requests)
In a nutshell, I was astonished that I had to modify the value to make imports of logs ranging from N-8 to N-1 (yesterday) possible. I don’t have any recollection to having to do this previously (I also didn’t find any documentation on it, but had to look into Core\Tracker\Request to find that it was necessary).
Here’s the snippet:
// If timestamp in the past, token_auth is required
$timeFromNow = $this->timestamp - $cdt;
$isTimestampRecent = $timeFromNow < $this->customTimestampDoesNotRequireTokenauthWhenNewerThan;
if (!$isTimestampRecent) {
if (!$this->isAuthenticated()) {
$message = sprintf("Custom timestamp is %s seconds old, requires &token_auth because it is bigger than %s...", $timeFromNow, $this->customTimestampDoesNotRequireTokenauthWhenNewerThan);
Common::printDebug($message);
Common::printDebug("WARN: Tracker API 'cdt' was used with invalid token_auth");
throw new InvalidRequestParameterException($message);
}
}
The other thing is naming. Looking at the snippet here above, you can see that it starts by calculating $timeFromNow, the age of the request to import. We then determine whether a record is recent by determining whether its age is younger (smaller) than the value set through tracking_requests_require_authentication_when_custom_timestamp_newer_than.
Value in global.php.ini/config.php.ini:
tracking_requests_require_authentication_when_custom_timestamp_newer_than = 86400;
IMHO, the variable is badly named. It should be named tracking_requests_require_authentication_when_custom_timestamp_older_than because a request’s timestamp (the difference to today actually) is compared to that value and authentication is required when the timestamp of the request is more than tracking_requests_require_authentication_when_custom_timestamp_newer_than seconds in the past.
Or am I getting this wrong and we’re supposed to use a token_auth value for log-analytics as per What is the token_auth and where can I find this token to use in the API calls? FAQ - Analytics Platform - Matomo ?