After upgrade to 2.5 import_logs.py fails 364 Field 'location_browser_lang'

Support & Bugs
1

I use the log import to import static files from our CDN servers and following the upgrade to 2.5 it no longer works.

2014/08/20 10:18:03 [error] 21463#0: *26 FastCGI sent in stderr: “PHP message: Error in Piwik (tracker): Error query: SQLSTATE[HY000]: General error: 1364 Field ‘location_browser_lang’ doesn’t have a default value In query: INSERT INTO piwik_log_visit (idvisitor, config_id, location_ip, custom_var_k1, custom_var_v1, config_browser_version, config_os, config_cookie, config_director, config_browser_name, referer_url, visit_total_searches, visit_total_actions, referer_keyword, referer_name, referer_type, config_flash, config_gears, visit_total_events, config_resolution, visitor_localtime, visitor_days_since_last, config_windowsmedia, config_silverlight, config_java, config_pdf, config_quicktime, config_realplayer, visit_exit_idaction_url, visit_exit_idaction_name, config_device_brand, config_device_model, config_device_type, location_country, config_os_version, idsite, visitor_count_visits, visitor_returning, visit_entry_idaction_name, visit_entry_idaction_url, visit_first_action_time, visitor_days_since_order, visit_total_time, visit_goal_buyer, visit_goal_convert” while reading response header from upstream,

If there a workaround for this problem ?

2

Thank you for the report! I created an issue at: Field 'location_browser_lang' doesn't have a default value In query · Issue #6028 · matomo-org/matomo · GitHub

please stay tuned on this ticket, we will try resolve ASAP

3

Is the UserSettings plugin maybe deactivated?

4

@thomas-piwik

UserSettings is active, we have a standard install with no plugins disabled.

5

Which PHP version are you using and which operating system?

6

And can you check if you see an entry like


Plugins_Tracker[] = "UserSettings"

in your config/configi.ini.php file?

7

@thomas-piwik

PHP 5.4.4-14+deb7u12 (cli) (built: Jun 30 2014 13:15:11)
Copyright © 1997-2012 The PHP Group
Zend Engine v2.4.0, Copyright © 1998-2012 Zend Technologies

Distributor ID: Debian
Description: Debian GNU/Linux 7.6 (wheezy)
Release: 7.6
Codename: wheezy

8

@thomas-piwik

Yes it appears active in the config file.

[Plugins_Tracker]
Plugins_Tracker[] = "Provider"
Plugins_Tracker[] = "Goals"
Plugins_Tracker[] = "PrivacyManager"
Plugins_Tracker[] = "UserCountry"
Plugins_Tracker[] = "Login"
Plugins_Tracker[] = "UsersManager"
Plugins_Tracker[] = "SitesManager"
Plugins_Tracker[] = "DevicesDetection"
Plugins_Tracker[] = "IPv6Usage"
Plugins_Tracker[] = "CoreHome"
Plugins_Tracker[] = "Actions"
Plugins_Tracker[] = "Referrers"
Plugins_Tracker[] = "UserSettings"
Plugins_Tracker[] = "Events"
Plugins_Tracker[] = "VisitTime"
Plugins_Tracker[] = “VisitorInterest”

9

Can you maybe try to run the log importer only with a few log lines and see if the error occurs as well? If so, can you maybe send me a few log lines where you know it fails to “thomas @ piwik.org”? So far it looks all good and I don’t see a reason why it should not work

10

@stbg what version of Mysql do you use? seems there was at least one bug in some previous mysql versions MySQL Bugs: #38173: " Field doesn't have a default value " with row-based replication

11

@matt

Server version: 5.6.20 MySQL Community Server (GPL)

12

@thomas-piwik

200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /img/close.png HTTP/1.1” 200 280 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /img/prev.png HTTP/1.1” 200 1360 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /img/next.png HTTP/1.1” 200 1350 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /fonts/fontawesome-webfont.ttf?v=4.0.3 HTTP/1.1” 301 178 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /fonts/glyphicons-halflings-regular.ttf HTTP/1.1” 200 41236 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /img/back.jpg HTTP/1.1” 200 134349 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /fonts/fontawesome-webfont.ttf HTTP/1.1” 200 80652 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
200.151.124.101 - - [19/Aug/2014:17:25:12 +0100] “GET /img/loading.gif HTTP/1.1” 200 8476 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36"
180.76.5.74 - - [20/Aug/2014:03:13:18 +0100] “GET /js/jquery.cookie.js HTTP/1.1” 200 1375 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.5.58 - - [20/Aug/2014:03:13:18 +0100] “GET /js/bootstrap.min.js HTTP/1.1” 200 7278 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.5.19 - - [20/Aug/2014:03:13:18 +0100] “GET /js/holder.js HTTP/1.1” 200 4553 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.6.132 - - [20/Aug/2014:03:13:18 +0100] “GET /js/lightbox-2.6.min.js HTTP/1.1” 200 2099 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.6.233 - - [20/Aug/2014:03:13:18 +0100] “GET /js/jquery.mixitup.min.js HTTP/1.1” 200 4676 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.5.74 - - [20/Aug/2014:03:13:18 +0100] “GET /js/app.js HTTP/1.1” 200 506 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.5.58 - - [20/Aug/2014:03:13:18 +0100] “GET /js/styleswitcher.js HTTP/1.1” 200 282 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.5.19 - - [20/Aug/2014:03:13:19 +0100] “GET /js/syntax/shCore.js HTTP/1.1” 200 8718 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.6.143 - - [20/Aug/2014:03:13:19 +0100] “GET /js/jquery-1.10.2.min.js HTTP/1.1” 200 32817 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.6.132 - - [20/Aug/2014:03:13:19 +0100] “GET /js/syntax/shBrushXml.js HTTP/1.1” 200 898 “https://www.thebritishgeek.co.uk/” "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
180.76.6.233 - - [20/Aug/2014:03:13:19 +0100] “GET /js/syntax/shBrushJScript.js HTTP/1.1” 200 756 “https://www.thebritishgeek.co.uk/” “Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)”

13

I think “location_browser_lang” is not present in the insert but marked as “not null” so it triggers an error. “location_browser_lang” is supposed to return a value but there might be one case where it does not return a value.

Can you try to apply this patch and test if it works? refs #6028 make sure there is always a value defined as the field is … · matomo-org/matomo@c01a9b8 · GitHub

In case you are using any opcache like the one builtin or APC make sure to reset it in case it doesn’t work.

14

@thomas-piwik

I have applied that and done a rest run on 1,000 lines all imported fine. I will now run against my main log files about 28 million lines and confirm back that no errors.

15

@thomas-pwik @matt

Thanks for your help today much appreciated, donation on the way next payday.