Log-analytics problem with w3c extended format

Support & Bugs
1

using the w3c extended format crashes for me when I added a custom column

My Command is as follows…

/home/weblogic/Python-2.7.8/python /home/weblogic/piwik-log-analytics/import_logs.py --url=http://su-soametricsprd01/piwik --idsite=1 --token-auth=8b4da1b22b12c2be757a0cf6db2811f3 --log-format-name=w3c_extended --w3c-map-field=cs(X-Forwarded-For)=c-ip /u01/app/oracle/product/ofusion/user_projects/domains/websrv/servers/logs/access.log

I’ve lost the error message, but is was at line 777 'if default_name not in type(format).fields:"

I changed the line to be " if default_name not in W3cExtendedFormat.fields: " Which fixed the issue.

Here’s a snippet of my log file


#Version:	1.0
#Fields:	date time cs-uri-stem cs-uri-query cs(X-Forwarded-For) cs(User-Agent) cs(Referer) sc-status sc-bytes cs-host cs-username time-taken
#Software:	WebLogic
#Start-Date:	2015-07-14	16:50:45
2015-07-14	16:50:45	/sale-history-service-2.0-GA/saleHistoryService/	-	"10.1.44.70"	"Apache-HttpClient/4.1.1 (java 1.5)"	-	200	-	-	-	0.716
2015-07-14	16:50:45	/sale-history-service-2.0-GA/saleHistoryService/	-	"10.1.44.70"	"Apache-HttpClient/4.1.1 (java 1.5)"	-	200	-	-	-	0.723

Am I doing something wrong ?

2

Hi there,

it is a bug. please create bug report with steps to reproduce (ie. log file sample) in the tracker for log analytics: Issues · matomo-org/piwik-log-analytics · GitHub

3

python3 ./import_logs.py --url=xxxx/matomo --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --log-format-name=w3c_extended --w3c-map-field [X-FORWARDED-FOR=c-ip] --idsite=2 /home/xxxx/u_ex240112_x.log
and it gives me error
Traceback (most recent call last):
File “./import_logs.py”, line 2766, in
config = Configuration()
File “./import_logs.py”, line 1123, in init
self._parse_args(self._create_parser(), argv)
File “./import_logs.py”, line 1085, in _parse_args
if default_name not in type(format).fields:
AttributeError: type object ‘builtin_function_or_method’ has no attribute ‘fields’

what is wrong

4

Hi @sabrykamel
It is better to create a new thread than reply to a very old one… (8 years old is quite old :thinking: )