Fatal error: HTTP Error 500 Internal Server Error when I try to use the script "import_logs.py"

Support & Bugs
1

Hello,

In advance thank you for your help, I’m beginner with Matomo and my goal is to upload every day an apache log access (500 000 lines) using the python script available on “/var/www/html/misc/log-analytics/import_logs.py”.

When I try to upload my apache access log with this command line:
python import_logs.py --url=https://mysite --token-auth=*************** --idsite=1 /tmp/access_log

It’s working during some minutes and after I getting the following error message each time:
2020-09-18 10:49:34,321: [INFO] Error when connecting to Matomo: HTTP Error 500: Internal Server Error
2020-09-18 10:49:34,323: [INFO] Max number of attempts reached, server is unreachable!
Fatal error: HTTP Error 500 Internal Server Error, response: {“status”:“error”,“tracked”:198,“invalid”:0,“invalid_indices”:[]}
You can restart the import of “/tmp/access_log” from the point it failed by specifying --skip=168390 on the command line.

I already try to fix it regarding this post on github " https://github.com/matomo-org/matomo/issues/13833"

#send the following command by mysql shell:
ALTER TABLE matomo_log_visit CHANGE visit_total_interactions visit_total_interactions MEDIUMINT(5) UNSIGNED NULL DEFAULT ‘0’;

But it’s does’t work for me.

My config:
OS: rhel8
Web Server: httpd-2.4.37-21
Matomo version: 3.14.1
MySQL version: 10.3.17-MariaDB
PHP version: 7.2.24

2

I have the very same issue!
You might have a look into your access.log at that position, mostly it’s a malformed URL, like SQL injection or other intrusion attempts via URL string. This is done mostly by automated tools (=> script kiddies).

The problem by matomo is, that import_logs has no feature to simply (== automatically) skip these errors and go on. That means, that cronjobs generating incomplete reports, and you have to restart the import_logs manually with the --skip=X feature, which is just a pita as admin.
Furthermore, if your logfile has more than just one of these entries, you have to restart the import several times which is pita^2.

To Matomo devs:
Please add a feature like “–skip-errors” to prevent incomplete import runs!!
If the import throws an error 500 (like shown above), simply skip that line and go on!

1 Like