2.8 - Fatal error: Internal Server Error on import_logs.py


#1

We upgraded from 2.3 to 2.8 with the db upgrade via the command line last night.
Today I get the the following:

time ./import_logs.py --url=http://piwik.gpc.edu/piwik --idsite=2 --recorders=8 --enable-http-errors --enable-http-redirects --enable-static --enable-reverse-dns /tmp/piwik21798 --enable-bots

Fatal error: Internal Server Error
You can restart the import of “/tmp/piwik21798” from the point it failed by specifying --skip=148800 on the command line.

Looking /var/log/httpd/error_log, I see nothing related to this. I had run it earlier without the ‘-enable-bots’, which gave
me problems in 2.3. It would tell me I could use --skip=4880.

Are there any other error logs I can look at? Any ideas or a way I can get more information on the ‘Internal Server Error’?

Thanks,
Douglas


#2

I forgot to mention, that if I rerun it, I get the same error:

time ./import_logs.py --url=http://piwik.gpc.edu/piwik --idsite=2 --recorders=8 --enable-http-errors --enable-http-redirects --enable-static --enable-reverse-dns /tmp/piwik21798 --enable-bots --skip=148800

Fatal error: Internal Server Error
You can restart the import of “/tmp/piwik21798” from the point it failed by specifying --skip=148800 on the command line.

still with no errors in error_log.

Thanks,
Douglas


#3

I made a small change to piwik.php:

//error_reporting(E_ALL | E_NOTICE);
error_reporting(E_ALL);
@ini_set(“display_errors”,“On”);

starting at line 29. I did not know what to change in the config.ini.php to turn this on, so I made this
mod to the source. I then ran the command:

time ./import_logs.py --url=http://piwik.gpc.edu/piwik --idsite=2 --recorders=8 --enable-http-errors --enable-http-redirects --enable-static --enable-reverse-dns /tmp/piwik21798 --enable-bots --skip=148800 >& /home/sa_webdev/err

The first or second time it came back with the before mentioned errror. This time, it is running. It ran for over
24 hours before my session disconnected and hung-up on it. The log file was 5.2M lines.

In the results, I noticed pageviews went from around 164k to 702k and bounce rate went from 30% to 34% and
average time on site quadroupled. Everything else seemed ok. I also noticed one of the httpd deamons continuosly
running and the access_log has a lot of:

10.30.224.57 - - [21/Oct/2014:07:21:25 -0400] “POST /piwik/piwik.php HTTP/1.1” 200 32 “-” “Piwik/LogImport”

lines in it. The error log sometimes has a php memory issue (we need to up that).

When I ran it against the new log today, I got the error:

Fatal error: Internal Server Error
You can restart the import of “/tmp/piwik25886” from the point it failed by spec
ifying --skip=811200 on the command line.

Any suggestions here on:

  1. how may backtrack my log for this site and only this site to a particular day?
  2. do I have to rebuild from scratch?
  3. what is causing these problems?

Thanks for any ideas!
Douglas