Piwik crashing 3 times a day since upgrade to v2.0.3


#1

We had been running Piwik v1.12 on a high volume intranet site for several months without problem, ~10k visits per day.
We recently upgraded to v2.0.3 and are now finding that Piwik is crashing 3 or 4 times a day.

Symptoms seem to be that occasionally, one of the CGI / FastCGI processes will lock up (continually 50% cpu) and cause multiple duplicate FastCGI processes to spawn. Within about 20 minutes there will be hundreds of CGI/FastCGI processes listed in task manager, and the IIS Application pool will finally crash giving error 500 – service unavailable.

We have set ’error_reporting=E_ALL’ in php.ini, however the only errors logged seem to be after the hundreds of processes are spawned.

Windows Event log displays numerous Application errors during this time period but don’t indicate what the cause might be:
[size=small]Faulting application name: php-cgi.exe, version: 5.4.14.0, time stamp: 0x5165bd29
Faulting module name: KERNEL32.DLL, version: 6.2.9200.16627, time stamp: 0x51a7dbea
Exception code: 0xc0000005
Fault offset: 0x00025f68
Faulting process id: 0x19ac
Faulting application start time: 0x01cf3204c99a6e26
Faulting application path: C:\Program Files (x86)\PHP\v5.4\php-cgi.exe
Faulting module path: C:\Windows\SYSTEM32\KERNEL32.DLL
Report Id: 07554832-9df8-11e3-9403-00155dbf990e
Faulting package full name:
Faulting package-relative application ID:[/size]

MySQL logs don’t show anything, and all tables pass a quick health check.

Piwik server is:
Windows Server 2012
IIS 8
PHP 5.4.14
8GB, Xeon E5-2650@2GHz
MySQL v5.6.15

Any advise you can give to track down the cause of the problem would br greatly appreciated.
Thanks


#2

check after upgrading piwik that the php.ini mem settings were not lowered im guessing for your site 512 to 1024 megs ram ideal.

also check version of fastcgi maybe a newer one might help here

one more link that is sort of reminiscent might help to check out


#3

Many thanks for your reply.
I have changed the memory_limit to 512M in php.ini but unfortunately we are still having problems

Fastcgi seems to be baked into IIS8 so I can’t see how to upgrade this.

The link you posted suggests disabling persistent DB connections, however this is for Microsoft SQL and not MySQL.
Given the high number DB queries, would this cause significant performance issues?

The PHP and IIS log files don’t seem to provide any indication of what may be causing the problem
Can I turn on extra debugging within Piwik itself?


#4

ahh ok im betting microsft is the problem

see this link here

particularly the update for fast cgi has this been applied? ms is notorious for having to have lots of updates…


(Matthieu Aubry) #5

please also upgrade to 2.1 as we fixed many issues regarding this


#6

Thanks both for your help, unfortunately we’re still having issues.
I’ve spent the last few days tweeking various PHP/MySQL/Fast-CGI/App Pool settings, but I still can’t get it to remain up for more than a couple hours, sometimes it’s down with 5mins of an iisrestart.

The link referrs to updates to IIS7 and Server 2008.
We are on IIS8 and Server 2012, it looks as though we’re up to date with any windows updates.

This might be an option, however it looks like 2.1 hasn’t been officially released yet and I doubt we’d get a CAB request approved for a Release Candidate version.
Do you know when 2.1 is likely to go live?


#7

can you turn off fast cgi to see if spped slows down but stability is at least maintained?


(Matthieu Aubry) #8

Upgrade to Piwik 2.1. Also upgrade your PHP as if it segfaults / crashes, it’s possible there’s a bug in PHP


(Saqib Ali) #9

Hi,

I’m facing the same problem

Event viewer log has reported following

Log Name: Application
Source: Application Error
Date: 30/04/2014 16:08:29
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: id28253
Description:
Faulting application name: php-cgi.exe, version: 5.3.21.0, time stamp: 0x50f709a3
Faulting module name: KERNEL32.DLL, version: 6.2.9200.16627, time stamp: 0x51a7dbea
Exception code: 0xc0000005
Fault offset: 0x00025faf
Faulting process id: 0x3ae4
Faulting application start time: 0x01cf647b52464a00
Faulting application path: d:\php\php-cgi.exe
Faulting module path: C:\WINDOWS\SYSTEM32\KERNEL32.DLL
Report Id: 491c79e6-d079-11e3-93fd-b8ca3af74ef3

I’m still finding solutions.

Thanks


#10

Your php version needs to be upgraded. At least 5.3.3 but now 5.4 or 5.5 are pretty main stream.


(Saqib Ali) #11

Thanks lesjokolat.

I have sort out the problem.

After removing wincache extention, php is working fine.

Kind Regards
Saqib Ali