Internal Server Error Importing Logs


#1

I get this error:
Parsing log /mnt/weblogarchive/intra/intraapps.log.2015-02-28-00:00.umod…
19046 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 47 records/sec (avg), 238 records/sec (current)
19301 lines parsed, 238 lines recorded, 39 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 33 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 29 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 26 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 23 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 21 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 19 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 18 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 238 lines recorded, 16 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 568 lines recorded, 37 records/sec (avg), 330 records/sec (current)
19301 lines parsed, 568 lines recorded, 35 records/sec (avg), 0 records/sec (current)
19301 lines parsed, 952 lines recorded, 55 records/sec (avg), 384 records/sec (current)
Fatal error: Internal Server Error

When trying to import a log file from AolServer in ncsa combined format. The import command I am using is:

python /opt/xampp/htdocs/misc/log-analytics/import_logs.py --url=http://localhost --idsite=1 --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --skip=16101 --log-format-name=ncsa_extended /mnt/weblogarchive/intra/intraapps.log.2015-02-28-00:00.umod

It keeps doing this and telling me to use --skip.

Any ideas? Thanks!


(Matthieu Aubry) #2

Hi there

please check your webserver error log file for an error message there?


#3

So this is what I found in my error log:
[Tue Mar 03 08:59:52 2015] [error] [client ::1] Error in Piwik (tracker): Error query: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction In query: UPDATE piwik_log_visit SET idvisitor = ?, visit_total_time = ?, visit_last_action_time = ?, visit_exit_idaction_name = ?, visit_exit_idaction_url = ?, visit_total_actions = visit_total_actions + 1 , custom_var_k1 = ?, custom_var_v1 = ? WHERE idsite = ? AND idvisit = ? Parameters: array ( 0 => ‘\x1f u/\xff8\xb5\xe3’, 1 => 5886, 2 => ‘2015-02-27 15:43:27’, 3 => ‘7360’, 4 => 7361, 5 => ‘Not-Bot’, 6 => ‘Mozilla/5.0 (Windows NT 6.1; WOW64; rv:35.0) Gecko/20100101 Firefox/35.0’, 7 => 1, 8 => 59, )
[Tue Mar 03 10:11:09 2015] [error] [client 172.31.28.18] Error in Piwik (tracker): Invalid idSite: ‘0’, referer: http://tracker.usurf.usu.edu/piwik.php?dp=index
[Tue Mar 03 10:11:09 2015] [error] [client 172.31.28.18] Error in Piwik (tracker): Invalid idSite: ‘0’

This is what I ran:
python /opt/xampp/htdocs/misc/log-analytics/import_logs.py --url=http://localhost --idsite=1 --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots --skip=16101 --log-format-name=ncsa_extended /mnt/weblogarchive/intra/intraapps.log.2015-02-28-00:00.umod

Here is an example of my log file:
172.31.35.11 - - [28/Oct/2014:00:04:52 -0600] “GET /SYSTEM/success HTTP/1.1” 200 7 “” “Mozilla/4.0 (compatible; MSIE+7.0; PA Server Monitor Service; )” 0.005456 "user_id=0"
17.161.97.198 - - [28/Oct/2014:00:04:53 -0600] “GET / HTTP/1.1” 200 14136 “” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36” 0.018469 "user_id=0"
17.161.97.198 - - [28/Oct/2014:00:04:53 -0600] “GET / HTTP/1.1” 200 14136 “” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36” 0.017599 "user_id=0"
17.161.97.198 - - [28/Oct/2014:00:04:54 -0600] “GET / HTTP/1.1” 200 14136 “” “Mozilla/5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D163 Safari/9537.53” 0.016164 "user_id=0"
17.161.97.198 - - [28/Oct/2014:00:04:54 -0600] “GET / HTTP/1.1” 200 14136 “” “Mozilla/5.0 (iPhone; CPU iPhone OS 7_1 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D163 Safari/9537.53” 0.015856 "user_id=0"
17.161.97.198 - - [28/Oct/2014:00:04:54 -0600] “GET /apple-touch-icon.png HTTP/1.1” 200 26 “” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36” 0.007258 "user_id=0"
17.161.97.198 - - [28/Oct/2014:00:04:55 -0600] “GET /dyru/favicon.ico HTTP/1.1” 200 36 “” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36” 0.004848 "user_id=0"
17.161.97.198 - - [28/Oct/2014:00:04:55 -0600] “GET /favicon.ico HTTP/1.1” 200 1150 “” “Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1623.0 Safari/537.36” 0.004525 “user_id=0”

I did not use idSite 0 so I am not sure why I get that error


(Matthieu Aubry) #4

Hi there,

please comment / watch this bug report: Tracker: Serialization failure: 1213 Deadlock found when trying to get lock · Issue #6398 · matomo-org/piwik · GitHub


#5

Based on the log sample I posted previously can you tell me if ncsa_extended is the right format?