High Traffic Site archiving not working :(


#1

Hi,

i am a developer and sysadmin and I am using piwik for about 3 months on a high traffic site.
In the last 3 days we had about 200.000 page impressions a day and the archive cronjob dosen’t work any more.

I tried it directly at the shell and get this error:

Archiving period = year for idsite = 1…
PHP Fatal error: Allowed memory size of 2122317824 bytes exhausted (tried to allocate 5 bytes) in /var/www/piwik/core/DataTable.php on line 969

Fatal error: Allowed memory size of 2122317824 bytes exhausted (tried to allocate 5 bytes) in /var/www/piwik/core/DataTable.php on line 969

Piwik is running on an extra server just for piwik with a single core xeon and 2 GB RAM, so I couldn’t give php more memory…

Seems to me, that the archiving-process has memory-leaks and isn’t optimized.

Any hints? Tried all the hints from the FAQs and the docs…

Thanks,
Jan


(Peterbo) #2

Well, 2122317824 bytes are just ~253 Mb - make sure you setup the memory limit for php and php cli to a higher value.

~Peter


#3

memory_limit = 2048M -> in both CLI and apache php.ini

and I can see that the process for archiving is consuming all memory so that the server is swapping…


#4

Well, 2122317824 bytes are 2024 MB, says google: Google :wink:


(Peterbo) #5

Oh indeed, I used the “Bit”-Row for the calculation.

Well, that’s not good indeed - are you running Apache/Nginx/Lighttpd on the server?


#6

I am running Apache, but want to switch to cherokee in the next weeks.


(Matthieu Aubry) #7

Unfortunately a known issue: Archiving Memory exhausted error - Archiving memory leaks · Issue #766 · matomo-org/matomo · GitHub

we will work on this problem in the next few months as more and more piwik users are reaching this problem


#8

[quote=matt]
Unfortunately a known issue: Archiving Memory exhausted error - Archiving memory leaks · Issue #766 · matomo-org/matomo · GitHub

we will work on this problem in the next few months as more and more piwik users are reaching this problem[/quote]

I read this and I will debug the archive process to find a fix.