AutoArchiving and Email-Reports Piwik 1.8.2

Hi,

recently I updated to piwik 1.8.2 and the new auto-archiving-php-script. On first of july a email report should have been send, which did not work out. Any idea what could be wrong in my setup? Regards, Rickky

The following error message was logged:


[2012-07-01 00:05:01] [3594d5c2] ---------------------------
[2012-07-01 00:05:01] [3594d5c2] INIT
[2012-07-01 00:05:01] [3594d5c2] Querying Piwik API at: http://MYHOSTNAME/index.php
[2012-07-01 00:05:01] [3594d5c2] Running as Super User: XXXXX
[2012-07-01 00:05:02] [3594d5c2] Notes
[2012-07-01 00:05:02] [3594d5c2] - Reports for today will be processed at most every 3600 seconds. You can change this value in Piwik UI > Settings > General Settings.
[2012-07-01 00:05:02] [3594d5c2] - Reports for the current week/month/year will be refreshed at most every 3600 seconds.
[2012-07-01 00:05:02] [3594d5c2] - Archiving was last executed without error 59 min 59s ago
[2012-07-01 00:05:02] [3594d5c2] Will process 1 websites with new visits since 59 min 59s , IDs: 1
[2012-07-01 00:05:02] [3594d5c2] ---------------------------
[2012-07-01 00:05:02] [3594d5c2] START
[2012-07-01 00:05:02] [3594d5c2] Starting Piwik reports archiving...
[2012-07-01 00:05:02] [3594d5c2] Archived website id = 1, period = day, Time elapsed: 0.306s
[2012-07-01 00:05:05] [3594d5c2] Archived website id = 1, period = week, 29617 visits, Time elapsed: 2.298s
[2012-07-01 00:05:05] [3594d5c2] Archived website id = 1, period = month, 69113 visits, Time elapsed: 0.459s
[2012-07-01 00:05:12] [3594d5c2] Archived website id = 1, period = year, 1047430 visits, Time elapsed: 6.623s
[2012-07-01 00:05:12] [3594d5c2] Archived website id = 1, today = 32 visits, 4 API requests, Time elapsed: 9.690s [1/1 done]
[2012-07-01 00:05:12] [3594d5c2] Done archiving!
[2012-07-01 00:05:12] [3594d5c2] ---------------------------
[2012-07-01 00:05:12] [3594d5c2] SUMMARY
[2012-07-01 00:05:12] [3594d5c2] Total daily visits archived: 32
[2012-07-01 00:05:12] [3594d5c2] Archived today's reports for 1 websites
[2012-07-01 00:05:12] [3594d5c2] Archived week/month/year for 1 websites. 
[2012-07-01 00:05:12] [3594d5c2] Skipped 3 websites: no new visit since the last script execution
[2012-07-01 00:05:12] [3594d5c2] Skipped 0 websites day archiving: existing daily reports are less than 3600 seconds old
[2012-07-01 00:05:12] [3594d5c2] Skipped 0 websites week/month/year archiving: existing periods reports are less than 3600 seconds old
[2012-07-01 00:05:12] [3594d5c2] Total API requests: 4
[2012-07-01 00:05:12] [3594d5c2] done: 1/1 100%, 32 v, 1 wtoday, 1 wperiods, 4 req, 9691 ms, no error
[2012-07-01 00:05:12] [3594d5c2] Time elapsed: 9.691s
[2012-07-01 00:05:12] [3594d5c2] ---------------------------
[2012-07-01 00:05:12] [3594d5c2] SCHEDULED TASKS
[2012-07-01 00:05:12] [3594d5c2] Starting Scheduled tasks... 
[2012-07-01 00:05:14] [3594d5c2] ERROR: Got invalid response from API request: http://MYHOSTNAME/index.php?module=API&method=CoreAdminHome.runScheduledTasks&format=csv&convertToUnicode=0&token_auth=80acc871a002227e1b3fda8f2c4646c1&trigger=archivephp. Response was ''
[2012-07-01 00:05:14] [3594d5c2] 
[2012-07-01 00:05:14] [3594d5c2] done
[2012-07-01 00:05:14] [3594d5c2] ---------------------------
[2012-07-01 00:05:14] [3594d5c2] SUMMARY OF ERRORS
[2012-07-01 00:05:14] [3594d5c2] Error: Got invalid response from API request: http://MYHOSTNAME/index.php?module=API&method=CoreAdminHome.runScheduledTasks&format=csv&convertToUnicode=0&token_auth=80acc871a002227e1b3fda8f2c4646c1&trigger=archivephp. Response was ''
[2012-07-01 00:05:14] [3594d5c2] 1 total errors during this script execution, please investigate and try and fix these errors
[2012-07-01 00:05:14] [3594d5c2] ERROR: 1 total errors during this script execution, please investigate and try and fix these errors. First error was: Got invalid response from API request: http://MYHOSTNAME/index.php?module=API&method=CoreAdminHome.runScheduledTasks&format=csv&convertToUnicode=0&token_auth=80acc871a002227e1b3fda8f2c4646c1&trigger=archivephp. Response was ''
Error in the last Piwik archive.php run: 
1 total errors during this script execution, please investigate and try and fix these errors. First error was: Got invalid response from API request: http://MYHOSTNAME/index.php?module=API&method=CoreAdminHome.runScheduledTasks&format=csv&convertToUnicode=0&token_auth=80acc871a002227e1b3fda8f2c4646c1&trigger=archivephp. Response was ''

Hi,

I’m still facing the issue.

Any Hint where to start tracing the problem?

Right now the manual sending of my pdf-reports per email via the GUI fails also:

[Wed Aug 01 12:51:40 2012] [error] [client xxx.xxx.xxx.xxx] PHP Fatal error: Call to undefined function imagecreatefrompng() in /var/lib/pro/piwik/libs/tcpdf/tcpdf.php on line 7892, referer: http://MYDOMAIN/index.php?module=PDFReports&action=index&idSite=1&period=day&date=yesterday

Regards,

Rikky

Call to undefined function imagecreatefrompng()

you need to enable PHP GD extension to generate images. We definitely should clarify the error message - are you using 1.8.2 ?

also p lease try to update to 1.8.3 beta: http://forum.piwik.org/read.php?2,91869

Hi Matt,

yes we are using 1.8.2. As far as I know, we have the PHP GD extension enabled. I’ll check this.

Regards,

Rikky

Hi again,

we’ve a test-system and a productive-system.

The email-sending works for the test-system but not for the productive system.

On both systems is gd installed. php-gd or php5-gd is not installed. Do I need php5-gd?

If so, why does it work on the test-system? The big difference between both systems is the amount of piwik-data. On test just a few clicks, on prod 2000 à day.

Can I check via phpinfo if my gd-setup is correct?

Rikky

Please check the phpinfo() for the section “gd” which has to state “GD Support: enabled”.´You have to make sure that the GD-Functions are available in PHP. So, yes, you need the php5-gd libs.

Hi Peter,

thanks for the response. In phpinfo I do not find a section “GD support”. The only thing related to GD are the following flags in the section Configure Command:

–with-gd=shared
–enable-gd-native-ttf
–without-gdbm

At the end of the post I append the beginning of phpinfo().

I can imagine, that the missing php5-gd is the problem. But that does not explain why the report-generation works on the test-system and just fails on the productive-system. Both systems are configured equally and both do not have php5-gd installed. In both phpinfos() no “GD Support: enabled”.

Is it possible, that the pdf-report-generation on the test-system does not use any gd-function-calls because there are very few data tracked by piwik?

However, I’ll try to install php5-gd on the machines and see how it looks then.

Regards, Rikky


phpinfo()
PHP Version => 5.2.6

System => Linux MYHOST 2.6.32.13-0.4-default #1 SMP 2010-06-15 12:47:25 +0200 x86_64
Build Date => Feb 12 2010 15:24:59
Configure Command => ‘…/configure’ ‘–prefix=/usr’ ‘–datadir=/usr/share/php5’ ‘–mandir=/usr/share/man’ ‘–bindir=/usr/bin’ ‘–with-libdir=lib64’ ‘–includedir=/usr/include’ ‘–sysconfdir=/etc/php5/cli’ ‘–with-config-file-path=/etc/php5/cli’ ‘–with-config-file-scan-dir=/etc/php5/conf.d’ ‘–enable-libxml’ ‘–enable-session’ ‘–with-mm’ ‘–with-pcre-regex=/usr’ ‘–enable-xml’ ‘–enable-simplexml’ ‘–enable-spl’ ‘–enable-filter’ ‘–disable-debug’ ‘–enable-inline-optimization’ ‘–disable-rpath’ ‘–disable-static’ ‘–enable-shared’ ‘–program-suffix=5’ ‘–with-pic’ ‘–with-gnu-ld’ ‘–with-system-tzdata=/usr/share/zoneinfo’ ‘–enable-cli’ ‘–with-pear=/usr/share/php5/PEAR’ ‘–enable-bcmath=shared’ ‘–enable-calendar=shared’ ‘–enable-ctype=shared’ ‘–enable-dbase=shared’ ‘–enable-dom=shared’ ‘–enable-exif=shared’ ‘–enable-ftp=shared’ ‘–enable-mbstring=shared’ ‘–enable-mbregex’ ‘–enable-pcntl=shared’ ‘–enable-posix=shared’ ‘–enable-shmop=shared’ ‘–enable-soap=shared’ ‘–enable-sockets=shared’ ‘–enable-sysvmsg=shared’ ‘–enable-sysvsem=shared’ ‘–enable-sysvshm=shared’ ‘–enable-tokenizer=shared’ ‘–enable-wddx=shared’ ‘–with-zlib=shared’ ‘–with-bz2=shared’ ‘–with-curl=shared’ ‘–with-gd=shared’ ‘–enable-gd-native-ttf’ ‘–with-xpm-dir=/usr’ ‘–with-freetype-dir=/usr’ ‘–with-png-dir=/usr’ ‘–with-jpeg-dir=/usr’ ‘–with-zlib-dir=/usr’ ‘–with-t1lib=/usr’ ‘–with-gettext=shared’ ‘–with-gmp=shared’ ‘–enable-hash=shared’ ‘–with-iconv=shared’ ‘–with-imap=shared’ ‘–with-kerberos’ ‘–with-imap-ssl’ ‘–enable-json=shared’ ‘–with-ldap=shared’ ‘–with-ldap-sasl=/usr’ ‘–with-libedit=shared,/usr’ ‘–with-mcrypt=shared’ ‘–with-mysql=shared,/usr’ ‘–with-mysql-sock=/var/lib/mysql/mysql.sock’ ‘–with-mysqli=shared’ ‘–with-ncurses=shared’ ‘–with-unixODBC=shared,/usr’ ‘–with-openssl=shared’ ‘–with-pgsql=shared,/usr’ ‘–with-pspell=shared’ ‘–with-snmp=shared’ ‘–with-xmlrpc=shared’ ‘–enable-xmlreader=shared’ ‘–enable-xmlwriter=shared’ ‘–with-xsl=shared’ ‘–with-tidy=shared,/usr’ ‘–enable-dba=shared’ ‘–with-db4=/usr’ ‘–without-gdbm’ ‘–with-cdb’ ‘–with-inifile’ ‘–with-flatfile’ ‘–enable-pdo=shared’ ‘–with-pdo_sqlite=shared,/usr’ ‘–with-pdo-mysql=shared,/usr’ ‘–with-pdo-pgsql=shared,/usr’ ‘–with-pdo-odbc=shared,unixODBC,/usr’ ‘–with-sqlite=shared,/usr’ ‘–enable-sqlite-utf8’ ‘–enable-zip=shared’ ‘–enable-suhosin=shared’ '–disable-cgi’
Server API => Command Line Interface
Virtual Directory Support => disabled
Configuration File (php.ini) Path => /etc/php5/cli
Loaded Configuration File => /etc/php5/cli/php.ini
Scan this dir for additional .ini files => /etc/php5/conf.d
additional .ini files parsed => /etc/php5/conf.d/ctype.ini,
/etc/php5/conf.d/curl.ini,
/etc/php5/conf.d/dom.ini,
/etc/php5/conf.d/hash.ini,
/etc/php5/conf.d/iconv.ini,
/etc/php5/conf.d/json.ini,
/etc/php5/conf.d/mysql.ini,
/etc/php5/conf.d/mysqli.ini,
/etc/php5/conf.d/pdo.ini,
/etc/php5/conf.d/pdo_mysql.ini,
/etc/php5/conf.d/suhosin.ini,
/etc/php5/conf.d/tokenizer.ini,
/etc/php5/conf.d/xmlreader.ini,
/etc/php5/conf.d/xmlwriter.ini,
/etc/php5/conf.d/zlib.ini

[…]

Hello,

I solved the GD-issue but I’m still fighting with the report-creation.

Here some of the error messages:

[Mon Aug 06 15:05:39 2012] [error] [client xxx.xxx.xxx.xxx] PHP Fatal error: Maximum execution time of 300 seconds exceeded in /var/lib/piwik/core/DataTable/Filter/Truncate.php on line 45, referer: http://MYHOST/index.php?module=PDFReports&action=index&idSite=1&period=month&date=2012-08-05&updated=1

[Mon Aug 06 10:54:25 2012] [error] [client xxx.xxx.xxx.xxx] PHP Fatal error: Allowed memory size of 805306368 bytes exhausted (tried to allocate 72 bytes) in /var/lib/piwik/core/DataTable/Manager.php on line 77, referer: http://MYHOST/index.php?module=PDFReports&action=index&idSite=1&period=day&date=yesterday&updated=1

I’ve set the memory size to 1024MB which did not work also.

I tried to set the exectuion time to 0 which should be unlimited, but this failed also:

[Mon Aug 06 15:59:11 2012] [error] [client xxx.xxx.xxx.xxx] PHP Fatal error: Maximum execution time of 0 seconds exceeded in /var/lib/piwik/core/DataTable/Manager.php on line 77, referer: http://MYHOST/index.php?module=PDFReports&action=index&idSite=1&period=day&date=yesterday

So for our setup I’ll stop creating and using reports with Piwik 1.8.2 and hope that it will work in a future version. (I am not allowed to try the 1.8.3. Beta due to policies)

Keep up the good work!

Rikky