Importing import_logs.py misses some log lines

Support & Bugs
1

Hi, I’m trying to import IIS logs to piwik with import_logs.py executed with the --enable-bots option.
But I can´t match the results in piwik with the content of thw log.
Although --enable-bots is enabled, I’m missing log lines from these IPs:
66.55.X.X
157.55.X.X (microsoft)
66.249.72.X (google bot)
131.253.24.7 (bing)
Can any of you help me with this? Thank you in advance!

2

It should work and track data. Can you submit a log file with one line that we can use to reproduce issue?

3

Here you have an example log line:

2013-09-17 01:28:21 127.0.0.1 GET /robots.txt - 80 - 131.253.24.7 HTTP/1.1 msnbot-media/1.1+(+http://search.msn.com/msnbot.htm) - 200 0 0 402 124

Thank you for your help Matt.
*Please note that I’ve modified our server IP, all other data in the log line is the original.

4

Do you get the same result?

python misc/log-analytics/import_logs.py …/test-log-file --url=http://localhost/piwik-master
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log …/test-log-file…
Fatal error: Cannot guess the logs format. Please give one using either the --log-format-name or --log-format-regex option

5

After inserting the header of the log file before that log line, I don’t get any log format error, the log import summary informs after analysis abour:


1 requests imported succesfully
0 requests were downloads
4 requests ignored:
    4 invalid log lines
...

The header is an standard IIS logfile header and seems like the following:


#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2013-09-17 22:00:59
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc-status sc-substatus sc-win32-status sc-bytes time-taken

So the 4 invalid log lines are the commented ones, but the other line (the one posted to you yesterday) is not imported to piwik.
Thank you again for your help Matt.