Error importing access logs

Support & Bugs
1

I’m attempting to import my site’s access logs, but the Python script is throwing an error.


[root@gobdg piwik]# python /var/www/vhosts/gobdg.org/piwik/misc/log-analytics/import_logs.py --url=http://stats.gobdg.org /var/www/vhosts/gobdg.org/statistics/logs/access_log --idsite=1 --recorders=4 --enable-http-errors --enable-httpd-redirects --enable-static --enable-bots
  File "/var/www/vhosts/gobdg.org/piwik/misc/log-analytics/import_logs.py", line 395
    level=logging.DEBUG if self.options.debug >= 1 else logging.INFO,
                         ^
SyntaxError: invalid syntax
2

what version of python?

3

Python 2.4.3

4

looks like its too old a version 2.7.3 is one a saw someone had success with.

http://forum.piwik.org/read.php?2,88825

5

You should definitely upgrade to Python 2.7.x for this. Actually you should check and upgrade your whole server as Python 2.4.3 is at least 6 years old and if your server hasn’t been updated since then it is likely to have a huge amount of vulnerabilities.

6

I was able to get Python 2.7.3 installed and ran the importer.

This is the result I got, but I’m not seeing any of the data in Piwik:


[root@gobdg logs]# python2.7 /var/www/vhosts/gobdg.org/piwik/misc/log-analytics/import_logs.py --url=http://stats.gobdg.org /var/www/vhosts/gobdg.com/statistics/logs/access_log --idsite=1 --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots
0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log /var/www/vhosts/gobdg.com/statistics/logs/access_log...
2194 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
2194 lines parsed, 574 lines recorded, 286 records/sec (avg), 574 records/sec (current)
2194 lines parsed, 899 lines recorded, 299 records/sec (avg), 325 records/sec (current)
2194 lines parsed, 1735 lines recorded, 433 records/sec (avg), 836 records/sec (current)
Purging Piwik archives for dates: 2012-11-28
To re-process these reports with your new update data, execute the piwik/misc/cron/archive.php script, or see: http://piwik.org/setup-auto-archiving/ for more info.

Logs import summary
-------------------

    2194 requests imported successfully
    26 requests were downloads
    0 requests ignored:
        0 invalid log lines
        0 requests done by bots, search engines, ...
        0 HTTP errors
        0 HTTP redirects
        0 requests to static resources (css, js, ...)
        0 requests did not match any known site
        0 requests did not match any requested hostname

Website import summary
----------------------

    2194 requests imported to 1 sites
        1 sites already existed
        0 sites were created:

    0 distinct hostnames did not match any existing site:



Performance summary
-------------------

    Total time: 4 seconds
    Requests imported per second: 461.68 requests per second
7

Ok here i am at a standstill but i do see this one line

To re-process these reports with your new update data, execute the piwik/misc/cron/archive.php script, or see: [piwik.org] for more info.

maybe this can help?

8

You need to run the archiving script. The python script only imports the log file into the database and does not process/archive the data.

9

What is the archiving script? Is that the cron job I have running every 5 minutes? Because that didn’t seem to do anything.