No data populated by crashed log import script


#1

I have tried importing data from an access log (which uses a customized format) containing data for one week, sept. 21 to sept. 28 (almost 1 month ago).
Since the log file is big (900M), I tried first using a 3 line sample - and it seemed to work: I got 2 page views.

Then I tried to import the big log file.
The time taken for import was going to be quite big, as the average number of rows processed was about 30/seconds.
Halfway through the night the ssh connection got closed, and the python script most likely crashed.

Looking at the web interface this morning, all I can see is still only 2 page views.

I tried dropping all archive tables and regenerating reports, but I see no difference.
Note: there is no piwik tracker on the site.

If I look at table size, the action, visit, and visit_action tables all contain many million rows.

Is there any reason for the data not showing up? Any tip for fixing this?

Thanks
Gaetano


(Matthieu Aubry) #2

Check your server error log in case it logged any during tracking?

Otherwise it should work. you can set several parameters to the script to increase / decrease recorders etc. maybe this helps you


#3

Apache error log is empty, access log looks like this:


127.0.0.1 - - [20/Oct/2014:00:34:50 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 301 "-" "Piwik/LogImport"
127.0.0.1 - - [20/Oct/2014:00:34:58 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 300 "-" "Piwik/LogImport"
127.0.0.1 - - [20/Oct/2014:00:34:58 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 301 "-" "Piwik/LogImport"
127.0.0.1 - - [20/Oct/2014:07:37:30 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 300 "-" "Piwik/LogImport"
127.0.0.1 - - [20/Oct/2014:07:37:30 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 301 "-" "Piwik/LogImport"
127.0.0.1 - - [20/Oct/2014:07:37:32 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 300 "-" "Piwik/LogImport"
127.0.0.1 - - [20/Oct/2014:07:37:32 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 301 "-" "Piwik/LogImport"
127.0.0.1 - - [20/Oct/2014:07:37:48 +0100] "POST /piwik/piwik.php HTTP/1.1" 200 300 "-" "Piwik/LogImport"