Piwik 2.0 segmentation fault / access violation critical bug!

Xeon · December 17, 2013, 9:51pm

Hi,

after updating Piwik to 2.0 I see many errors like this:

Host: Fedora 20, PHP 5.5.6, Apache 2.4

[Tue Dec 17 17:18:20.819459 2013] [core:notice] [pid 4746] AH00052: child pid 4747 exit signal Segmentation fault (11)
[Tue Dec 17 17:18:20.820071 2013] [core:notice] [pid 4746] AH00052: child pid 4750 exit signal Segmentation fault (11)
[Tue Dec 17 17:18:50.890390 2013] [core:notice] [pid 4746] AH00052: child pid 4748 exit signal Segmentation fault (11)
[Tue Dec 17 17:18:50.890812 2013] [core:notice] [pid 4746] AH00052: child pid 4770 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:20.960116 2013] [core:notice] [pid 4746] AH00052: child pid 4787 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:20.960427 2013] [core:notice] [pid 4746] AH00052: child pid 4790 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:50.017266 2013] [core:notice] [pid 4746] AH00052: child pid 5127 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:51.019071 2013] [core:notice] [pid 4746] AH00052: child pid 4788 exit signal Segmentation fault (11)
[Tue Dec 17 17:20:20.078687 2013] [core:notice] [pid 4746] AH00052: child pid 4853 exit signal Segmentation fault (11)
[Tue Dec 17 17:20:50.139520 2013] [core:notice] [pid 4746] AH00052: child pid 4749 exit signal Segmentation fault (11)
[Tue Dec 17 17:20:50.139670 2013] [core:notice] [pid 4746] AH00052: child pid 5074 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:20.201130 2013] [core:notice] [pid 4746] AH00052: child pid 4751 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:20.201324 2013] [core:notice] [pid 4746] AH00052: child pid 5091 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:50.265777 2013] [core:notice] [pid 4746] AH00052: child pid 5177 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:50.266129 2013] [core:notice] [pid 4746] AH00052: child pid 5225 exit signal Segmentation fault (11)
[Tue Dec 17 17:25:46.745843 2013] [core:notice] [pid 4746] AH00052: child pid 4772 exit signal Segmentation fault (11)

This causes also other php applications running on this server to segfault!

Here is the same issue (with a different error message) on a CentOS 6.5 Host with PHP 5.3.3 and Apache 2.2:

[Tue Dec 17 13:10:50 2013] [error] [client IP] PHP Fatal error: Allowed memory size of 2147483648 bytes exhausted (tried to allocate 523800 bytes)

(PHP Memory Limit = 2 GB and the host has enough free memory)

Other php applications crashing because of the Piwik 2.0 access violation!

matthieu · December 18, 2013, 12:19am

Update your PHP to latest version as it’s very old and has so bugs that could cause this message. cheers

Xeon · December 18, 2013, 1:34am

[quote=matt]
Update your PHP to latest version as it’s very old and has so bugs that could cause this message. cheers[/quote]

PHP 5.5.6 is very old??? I’ve tested it with Fedora 20 PHP 5.5.6 and also with CentOS 6.5 PHP 5.3.3. Please note that Red Hat Enterprise Linux and also CentOS (10 years support) is based on PHP 5.3.3.

On both systems PHP 5.5.6 (Fedora) and 5.3.3 (CentOS), Piwik 2.0 causes memory access violations and segfaults.

matthieu · December 18, 2013, 4:12am

Sorry I misread the version number (thought it was 5.3.6)

are you using the cron How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo
are you using a PHP cache as eAccelerator, APC, xcache, maybe try disable just in case?

maybe this helps

ageltixx · December 19, 2013, 3:43pm

Same problem here. No cron, no caching.

PHP 5.3.3-7+squeeze18 (Debian)
Apache 2.2.16

I am now back on 1.12 what works fine and at this moment I don’t want to try to use 2.0 / 2.0.1 again, because 2.0 kills other Websites.

matthieu · December 19, 2013, 8:21pm

are you also using php 5.5.6 ? maybe try to update to latest php? we cannot reproduce the seg fault error so should be caused by php bug or so (seg fault errors are very rarely caused by bugs in our php code!)

ageltixx · December 19, 2013, 8:32pm

AFAIK is PHP 5.5 not available for Debian Squeezy. And I will not upgrade the whole server to wheezy just to enable PIWIK. Version 1.12 is running with no problems and 2.0 has massive memory- and segfault-problems. Is this a PHP-Error?

In addition: Xeon has Segfault in 5.5.6, see above.

Xeon · December 19, 2013, 11:28pm

[quote=ageltixx]
AFAIK is PHP 5.5 not available for Debian Squeezy. And I will not upgrade the whole server to wheezy just to enable PIWIK. Version 1.12 is running with no problems and 2.0 has massive memory- and segfault-problems. Is this a PHP-Error?

In addition: Xeon has Segfault in 5.5.6, see above.[/quote]

It’s not a PHP error. I have the segfault also with latest PHP 5.5.7 (fedora-testing repo).

Steps to reproduce:

Install Fedora 20 with Apache, PHP and MySQL
Install a new clean (new database) Piwik 2.x installation
Login to Piwik
Install a new clean Joomla 2.5 installation

The result is:
[Tue Dec 17 17:18:20.819459 2013] [core:notice] [pid 4746] AH00052: child pid 4747 exit signal Segmentation fault (11)
[Tue Dec 17 17:18:20.820071 2013] [core:notice] [pid 4746] AH00052: child pid 4750 exit signal Segmentation fault (11)
[Tue Dec 17 17:18:50.890390 2013] [core:notice] [pid 4746] AH00052: child pid 4748 exit signal Segmentation fault (11)
[Tue Dec 17 17:18:50.890812 2013] [core:notice] [pid 4746] AH00052: child pid 4770 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:20.960116 2013] [core:notice] [pid 4746] AH00052: child pid 4787 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:20.960427 2013] [core:notice] [pid 4746] AH00052: child pid 4790 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:50.017266 2013] [core:notice] [pid 4746] AH00052: child pid 5127 exit signal Segmentation fault (11)
[Tue Dec 17 17:19:51.019071 2013] [core:notice] [pid 4746] AH00052: child pid 4788 exit signal Segmentation fault (11)
[Tue Dec 17 17:20:20.078687 2013] [core:notice] [pid 4746] AH00052: child pid 4853 exit signal Segmentation fault (11)
[Tue Dec 17 17:20:50.139520 2013] [core:notice] [pid 4746] AH00052: child pid 4749 exit signal Segmentation fault (11)
[Tue Dec 17 17:20:50.139670 2013] [core:notice] [pid 4746] AH00052: child pid 5074 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:20.201130 2013] [core:notice] [pid 4746] AH00052: child pid 4751 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:20.201324 2013] [core:notice] [pid 4746] AH00052: child pid 5091 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:50.265777 2013] [core:notice] [pid 4746] AH00052: child pid 5177 exit signal Segmentation fault (11)
[Tue Dec 17 17:21:50.266129 2013] [core:notice] [pid 4746] AH00052: child pid 5225 exit signal Segmentation fault (11)
[Tue Dec 17 17:25:46.745843 2013] [core:notice] [pid 4746] AH00052: child pid 4772 exit signal Segmentation fault (11)

lesjokolat · December 19, 2013, 11:52pm

On your apache server the module below can you turn this off and try to see if it helps?

reqtimeout_module

Also had another idea once a while back the suhosin memory needed to be increased. Could this be applicable to your setup?

peterbo · December 20, 2013, 12:16am

Is there another Bytecode cache running in parallel of your PHP 5.5 opCode cache? (Like APC,xCache, etc.)

Which Apache-Modules are running? (apache2ctl -t -D DUMP_MODULES)
Which PHP Modules are running? (php -m)

Xeon · December 20, 2013, 11:09am

[quote=lesjokolat]
On your apache server the module below can you turn this off and try to see if it helps?

reqtimeout_module[/quote]

Turning off this module doesn’t help.

Also had another idea once a while back the suhosin memory needed to be increased. Could this be applicable to your setup?

No, I’m not using suhosin.

Xeon · December 20, 2013, 11:10am

[quote=“Peter Boehlke”]
Is there another Bytecode cache running in parallel of your PHP 5.5 opCode cache? (Like APC,xCache, etc.) [/quote]

No.

Which Apache-Modules are running? (apache2ctl -t -D DUMP_MODULES)
Which PHP Modules are running? (php -m)

httpd -v

Server version: Apache/2.4.6 (Fedora)
Server built: Oct 31 2013 17:11:47

php -v

PHP 5.5.7 (cli) (built: Dec 11 2013 06:36:38)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2013 Zend Technologies

httpd -t -D DUMP_MODULES

Loaded Modules:
core_module (static)
so_module (static)
http_module (static)
access_compat_module (shared)
actions_module (shared)
alias_module (shared)
allowmethods_module (shared)
auth_basic_module (shared)
auth_digest_module (shared)
authn_anon_module (shared)
authn_core_module (shared)
authn_dbd_module (shared)
authn_dbm_module (shared)
authn_file_module (shared)
authn_socache_module (shared)
authz_core_module (shared)
authz_dbd_module (shared)
authz_dbm_module (shared)
authz_groupfile_module (shared)
authz_host_module (shared)
authz_owner_module (shared)
authz_user_module (shared)
autoindex_module (shared)
cache_module (shared)
cache_disk_module (shared)
data_module (shared)
dbd_module (shared)
deflate_module (shared)
dir_module (shared)
dumpio_module (shared)
echo_module (shared)
env_module (shared)
expires_module (shared)
ext_filter_module (shared)
filter_module (shared)
headers_module (shared)
include_module (shared)
info_module (shared)
log_config_module (shared)
logio_module (shared)
macro_module (shared)
mime_magic_module (shared)
mime_module (shared)
negotiation_module (shared)
remoteip_module (shared)
rewrite_module (shared)
setenvif_module (shared)
slotmem_plain_module (shared)
slotmem_shm_module (shared)
socache_dbm_module (shared)
socache_memcache_module (shared)
socache_shmcb_module (shared)
status_module (shared)
substitute_module (shared)
suexec_module (shared)
unique_id_module (shared)
unixd_module (shared)
userdir_module (shared)
version_module (shared)
vhost_alias_module (shared)
dav_module (shared)
dav_fs_module (shared)
dav_lock_module (shared)
lua_module (shared)
mpm_prefork_module (shared)
proxy_module (shared)
lbmethod_bybusyness_module (shared)
lbmethod_byrequests_module (shared)
lbmethod_bytraffic_module (shared)
lbmethod_heartbeat_module (shared)
proxy_ajp_module (shared)
proxy_balancer_module (shared)
proxy_connect_module (shared)
proxy_express_module (shared)
proxy_fcgi_module (shared)
proxy_fdpass_module (shared)
proxy_ftp_module (shared)
proxy_http_module (shared)
proxy_scgi_module (shared)
systemd_module (shared)
cgi_module (shared)
php5_module (shared)

php -m

[PHP Modules]
bz2
calendar
Core
ctype
curl
date
dom
ereg
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
imap
intl
json
libxml
mbstring
mcrypt
mhash
mysql
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
readline
Reflection
session
shmop
SimpleXML
sockets
SPL
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tokenizer
wddx
xml
xmlreader
xmlrpc
xmlwriter
xsl
zlib

[Zend Modules]

lesjokolat · December 20, 2013, 10:38pm

hi what version of python is installed?

Xeon · December 20, 2013, 11:26pm

Update:

After upgrading to PiWik 2.0.2 no more memory/segfault errors!

So I think that the error has been fixed? I’m not able to reproduce it on 2.0.2!

@PiWik Developers:

Thank you!

Can you tell me the bugfix commit? Thanks.

matthieu · December 21, 2013, 5:26am

[quote=Xeon]
Update:

After upgrading to PiWik 2.0.2 no more memory/segfault errors!

So I think that the error has been fixed? I’m not able to reproduce it on 2.0.2!

@PiWik Developers:

Thank you!

Can you tell me the bugfix commit? Thanks.[/quote]

Im surprised the bug is fixed, I didn’t expect it. Maybe this fixed it: libxml_disable_entity_loader() breaks other softwares on same server · Issue #4409 · matomo-org/matomo · GitHub

ageltixx · December 21, 2013, 8:57am

Excellent news. I tried the Update, got “PHP Fatal error: Interface ‘Piwik\View\ViewInterface’ not found” in the logs, but Piwik 2.0.2 worked after that. No Segfaults. Thank you very much.

Xeon · December 21, 2013, 1:40pm

[quote=matt]
Im surprised the bug is fixed, I didn’t expect it. Maybe this fixed it: libxml_disable_entity_loader() breaks other softwares on same server · Issue #4409 · matomo-org/matomo · GitHub

Hi Matt,

You’re right. I have tested it for you and I can confirm that commit cb05112137c0a60d4bcb8e6c46b6fd9782bc3936 fixed it.

Thank you.

Xeon · December 21, 2013, 1:44pm

[quote=matt]
Im surprised the bug is fixed, I didn’t expect it. Maybe this fixed it: libxml_disable_entity_loader() breaks other softwares on same server · Issue #4409 · matomo-org/matomo · GitHub

Hi Matt,

Yes, this is it. I have tested it for you. Commit cb05112137c0a60d4bcb8e6c46b6fd9782bc3936 (Fixes #4409 Removing the function call as it is not thread safe and b… · matomo-org/matomo@cb05112 · GitHub) fixed it.

Thank you.