Email-reports: Out of memory


#1

Using my email-reports with 1.7 results are errors:

Internal Server Error

or error
Fatal error: Out of memory (allocated 24903680) (tried to allocate 2280 bytes) in /homepages/4xxxxx/core/ReportRenderer.php on line 202
or
…piwik/libs/pChart2.1.3/class/pImage.class.php on line 93

if I disable ImageGraph-plugin it works fine - but wihthout grafics in the reports


(Matthieu Aubry) #2

Your PHP memory limit is very low, please increase it: Troubleshooting - Analytics Platform - Matomo


(Fellmer Lloyd) #3

Hi matt

I also get the “Out of memory” error while rendering email reports (PDF).
This happens for all reports using data for a whole month or more.

I’ve increased the PHP memory limit to 2GB - still not working.
The script consumes the whole 2GB (and fails).

I think this is a design issue.
Are there any plans to change this (e.g. calculate in the DB )?

It would be a nice feature for Piwik.

Regards,
Fellmer


#4

Problem also after increasing PHP memory - version 1.7.1.

In some cases a report “weekly” fail BUT same as a range (from-to) works. Reports for other websites - with much more data work.

I think this is a design issue, too.
Others with the same problem or a work aoround?

regards
salguri


(Fellmer Lloyd) #5

That’s strange… I don’t get any Report working for a week on this site.
A range leads to the same error.

I’ll try to export the data to a functioning reporting system as a workaround.
However, I’ll check google analytics for this feature (i.e. working reports).

Regards,
Fellmer


#6

I’ve not been able to send email reports since the upgrade either. Have increased time-outs from 300(ish) to 500 seconds, and now get the memory issue

PHP Fatal error: Allowed memory size of 536,870,912 bytes exhausted (tried to allocate 3 bytes)


#7

this is crazy, I don’t think I can go higher than 1024M on this server, and it appear towards the end it was taking memory off the mySQL instance

PHP Fatal error: Allowed memory size of 1,073,741,824 bytes exhausted (tried to allocate 40 bytes)


(Fellmer Lloyd) #8

I’ve tried it with 2 and 4 gigabytes. Still not working.
As I said, it’s a design issue.

Actually, there’s already a ticket for this issue.
However, it always gets reassigned to a future version.


#9

but the reporting was working fine in the previous version for us.

I’m tempted to reload the older version since the reporting is pretty much the main feature we use, being able to send users recent stats


(Matthieu Aubry) #10

Maybe the graphs are causing the problem?

I can’t think of anything else that has changed.

try disabling the graph with this patch:


Index: plugins/PDFReports/API.php
===================================================================
--- plugins/PDFReports/API.php	(revision 5891)
+++ plugins/PDFReports/API.php	(working copy)
@@ -370,7 +370,7 @@
 										$aggregateReportsFormat == Piwik_PDFReports::AGGREGATE_REPORTS_FORMAT_TABLES ||
 										$aggregateReportsFormat == Piwik_PDFReports::AGGREGATE_REPORTS_FORMAT_TABLES_GRAPHS;
 
-			$report['displayGraph'] = 	(!$isAggregateReport ||
+			$report['displayGraph'] = 	false && (!$isAggregateReport ||
 										 $aggregateReportsFormat == Piwik_PDFReports::AGGREGATE_REPORTS_FORMAT_GRAPHS ||
 										 $aggregateReportsFormat == Piwik_PDFReports::AGGREGATE_REPORTS_FORMAT_TABLES_GRAPHS )
 										 && Piwik::isGdExtensionEnabled()

Does the memory requirement go down? is it faster?

Thanks for your help. It would be nice if we could find the problem !


#11

I’ve made that code change, but the timeout still exists (not the memory problem). Fiddling with the report, it seems the Referrers section is causing the problem. If I put everything else in, except this area, the report runs. If I only put this area in, I can not include all the options without the error, but I can individually or in groups. It seems the Keywords/websites report causes the problem, if this helps.

To summarise, I’ve now got the graphing back on, with everything turned on for the report (PDF, tables/graphs for all) minus these reports:

dashboard
keywords
continent
custom
provider
ALL visitor settings

and it works almost instantly. Is there a resource issue in the “Referrers” section?

many thanks for your help on this


#12
  • try to select RAGE instead of WEEK for the period to be reported
  • try to select DOWNLOAD instead of SEND MAIL

in SOME cases that works - or report is already sent by mail but piwik report generator dont finish work


selcted 6 reports to be send by mail weekly, but only 2 or 3 will be sent - that problem exists also with earlier versions


#13

I’m just trying to recreate the old reports I had on there that I removed, and now I’m finding the report that was just working is not -so it seems intermittent. Have been testing with download rather than emailing, since it’s a pretty simple task attaching the PDF to an email. I don’t think the problem is there.

I’ve now got the graphs back off, and it seems to be outlinks/downloads that are causing the problem

what would be a sensible memory/time out limit? is 500mb/300secs too tight? It seems like a fair bit to me, for this task?


#14

if the hints mentioned above dont work

  • enlarge PERIOD by one day or
  • try to deselect KEYWORDS

… still: It is an ERROR!


#15

will continue to play with it when I get the time, but for now, with reduced reporting and removed graphics it’s doing a job of sorts.

I find it worrying that it either working within (say) 10 seconds, or fails with timeouts over 300 seconds/out of memory. There’s no middle ground which you’d expect when tweaking the reports chosen?

I don’t see how messing with the ‘period’ will change this


(Matthieu Aubry) #16

I really have no idea what the problem could be. Make sure you upgrade PHP and apache to latest version> I also recommend to setup: How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo
if you find more info or a bug, let us know here!


#17

ok great thanks. This is PHP on windows, so that could be doing all sorts different.

I find the different between working/failing so clear cut rather than gradual time increase weird.

is there a general standard for timeout/memory setting within PHP ini, or is this completely custom depending on setup?

my boss has also had trouble going back and doing monthly reports with timeouts -I thought the auto-archiving helped this process? Or does the auto-archiving only digest the visitor log, and not produce any numbers, so to speak?

is there any mySQL mainenance I should be doing? I’m from an MS SQL background and not sure what goes on in the mySQL box!

if I find out anything interesting, I will let you know! thanks


(Matthieu Aubry) #18

If you setup archive.php it should process all monthly report, so your boss shouldn’t have to wait or experience issue at all.
However, make sure you use this patched archive.php which fixes a bug: http://dev.piwik.org/trac/export/5927/trunk/misc/cron/archive.php


#19

archiving is setup and runs hourly. Have just applied the patch, but seemed to be working OK?:

[2012-03-07 09:32:02] [486031be] Done archiving!
[2012-03-07 09:32:02] [486031be] ---------------------------
[2012-03-07 09:32:02] [486031be] SUMMARY
[2012-03-07 09:32:02] [486031be] Total daily visits archived: 6351
[2012-03-07 09:32:02] [486031be] Archived today’s reports for 31 websites
[2012-03-07 09:32:02] [486031be] Archived week/month/year for 31 websites.
[2012-03-07 09:32:02] [486031be] Skipped 7 websites: no new visit since the last script execution
[2012-03-07 09:32:02] [486031be] Skipped 0 websites day archiving: existing daily reports are less than 3600 seconds old
[2012-03-07 09:32:02] [486031be] Skipped 0 websites week/month/year archiving: existing periods reports are less than 3600 seconds old
[2012-03-07 09:32:02] [486031be] Total API requests: 124
[2012-03-07 09:32:02] [486031be] done: 31/31 100%, 6351 v, 31 wtoday, 31 wperiods, 124 req, 46680 ms, no error
[2012-03-07 09:32:02] [486031be] Time elapsed: 46.680s
[2012-03-07 09:32:02] [486031be] ---------------------------
[2012-03-07 09:32:02] [486031be] SCHEDULED TASKS
[2012-03-07 09:32:02] [486031be] Starting Scheduled tasks…
[2012-03-07 09:32:03] [486031be] No task to run
[2012-03-07 09:32:03] [486031be] done


(Fellmer Lloyd) #20

We’re running Piwik on Windows/IIS7 as well.
Maybe that’s the root of the problem.

I use a cron job as well and cannot get any report working (>= 1 Week, dosen’t realy matter which reports).

Sadly I cannot test further right now - I’ve got other projects as well.
But if you need some information (system etc.) to find the error please let me know.