Log analytics regex formatting help!

General questions
1

Hello! I’ve used Piwik in the past with the JS tracking code, but I’m currently looking at setting it up to track activity on our ezproxy server via the server logs. Currently I do this using AWStats but want the rich dashboard experience of Piwik. I’ve set up a test site on my localhost and pulled over a .log file to play with. The logfile format in AWStats is:

%host %other %other %logname %other %time1 %methodurl %code %bytesd

but I’m not sure how to translate this to a Regex for Piwik. I’ve tried running the file with autodetect but repeatedly get Fatal error: cannot automatically determine the log format using the first 100000 lines of the log file.

Any pointers?

2

Hello again! So, I’ve done a bit more research on this and come up with the following Regex:

(?P[\w-.])(?::\d+)? \S+ \S+ \S+ \S+ [(?P.?) (?P.?)] “(?P\S+) (?P.*?) (?P\S+)” (?P\S+)? (?P\S+)

Which turns up the following error:

File “import_logs.py”, line 2314, in
config = Configuration()
File “import_logs.py”, line 864, in init
self._parse_args(self._create_parser())
File “import_logs.py”, line 802, in _parse_args
self.format = RegexFormat(‘custom’, self.options.log_format_regex, self.options.log_date_format)
File “import_logs.py”, line 183, in init
self.regex = re.compile(regex)
File “c:\Python27\lib\re.py”, line 194, in compile
return _compile(pattern, flags)
File “c:\Python27\lib\re.py”, line 251, in _compile
raise error, v # invalid expression
sre_constants.error: unknown specifier: ?P[

3

this part is not valid and causes the error sre_constants.error: unknown specifier: ?P[

4

Hi! Thanks for getting back to me; how would you suggest I change this up? Sorry if I sound dim, I’m not overly well versed in Regex. Our log file format is (anonymised):

1.1.1.1 - abcdef 123456 % [01/Jul/2015:00:00:00 +0000] “GET http://a.url.com HTTP/1.1” 304 0

5

Hi folks, any thoughts on this? I’ve tried a few variations but I can’t find any kind of useful documentation on how to translate this so any pointers on how I can form this Regex would be brilliant!

6

Please ask a developer or someone who knows Regex and if you find the answer let us know how we can improve our docs so other people don’t have the same issue :+1:

7

Dear Andrew B.

A cordial greeting.

I discovered Motomo (Piwik) and I do not know how to read my ezproxy log properly.

From what I have read since 2016, you have worked with this theme.

I want to ask you to help me shed light on this issue.

regards