Log-analytics & import_logs.py

Hi,

I try to import some logs from the nginx webserver.

I have followed the howto from https://github.com/piwik/piwik/tree/master/misc/log-analytics#setup-nginx-logs.

This setup generates a nice access log file that looks like:

{“ip”: “192.168.1.10”,“host”: “www.test.nl”,“path”: “/”,“status”: “200”,“referrer”: “https://www.test.nl/","user_agent”: “Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36”,“length”: 915,“generation_time_milli”: 0.001,“date”: “2015-01-02T13:20:31+01:00”}

When I try to import this access log with:
$ python /var/www/misc/log-analytics/import_logs.py --url=http://ka-webanalyse-01.edu.local/ --add-sites-new-hosts --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --log-format-name=nginx_json --config /var/www/config/config.ini.php access.log

I get the flowing error:

0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log access.log…
Traceback (most recent call last):
File “/var/www/misc/log-analytics/import_logs.py”, line 1750, in
main()
File “/var/www/misc/log-analytics/import_logs.py”, line 1717, in main
parser.parse(filename)
File “/var/www/misc/log-analytics/import_logs.py”, line 1576, in parse
resolver.check_format(format)
File “/var/www/misc/log-analytics/import_logs.py”, line 1119, in check_format
elif ‘host’ not in format.regex.groupindex and not config.options.log_hostname:
AttributeError: ‘NoneType’ object has no attribute ‘groupindex’

As I understands is correctly is it missing the host parameter in the access log but as you can see it is there. What goos wrong.

Kinds regards,

Michiel Piscaer

Hi there, thanks for the report. do you mind creating an issue on our issue tracker? we will investigate: Issues · matomo-org/piwik · GitHub

Same error with the format_name option w3c_extended.

Like matt asked, I have created an issue on github: log-analytics & import_logs.py, hosts parameter failing · Issue #6919 · matomo-org/matomo · GitHub

Thanks for that!