Errors when importing large log files

Support & Bugs
1

This is my 3rd time trying to submit this topic. Prior ones get flagged a spam. I’m guessing because I’ve included log file rows. I will try to sanitize them enough that this post wont get flagged.

When trying to import large apache log files (380,000 rows) it fails at about 80,000 rows. This is during a scripted import where many log files are loaded. This is by far the largest.

I see 500 errors from the import python script.

In the PHP logs I see the rows below.

RHEL 6 Apache 2.4 PHP 5.6 Python 2.6.6 MySQL 5.6.26

-Kyle

[13-Nov-2015 06:57:08 UTC] Error in Piwik (tracker): Error query: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column ‘visit_total_actions’ at row 1

In query: UPDATE piwik_log_visit SET idvisitor = ?, visit_total_time = ?, visit_last_action_time = ?, visit_exit_idaction_url = ?,

visit_total_actions = visit_total_actions + 1 , custom_var_k1 = ?, custom_var_v1 = ? WHERE idsite = ? AND idvisit = ?

Parameters: array ( 0 => ‘^<91>~P^Bk^D’, 1 => 19067, 2 => ‘2015-11-12 10:17:46’, 3 => 488738, 4 => ‘sessionid’, 5 => ‘62eba080.524389da9f587’, 6 => 13, 7 => 111924, )

2

After some review and confirmed by some nice PMs, it appears that we have a VERY busy user (likely a monitor) that has exceeded the maximum of a SMALLINT type in mySQL.

So we can do an ALTER on this table to bring this field to a type INT. Something like: “ALTER TABLE piwik_log_visit MODIFY visit_total_actions INT UNSIGNED;”

Can this be made a part of the Piwik standard code? Should I submit this is a request or a bug?

If I change the field myself, will future upgrades break or revert my change?

Thank you!
-Kyle

3

Hi there,

You are experiencing this bug: Numeric value out of range: 1264 Out of range value for column 'visit_total_actions' at row 1 In query · Issue #6834 · matomo-org/piwik · GitHub