Email Reports will not be sent

Support & Bugs
1

Hi!

I have a problem with the e-mail reports. The manual sending of reports via Matomo interface works, but the automatic sending via Archive Cron Job does not work.

Log:

INFO [2018-11-01 15:42:31] 79180  Scheduler: executing task Piwik\Plugins\ScheduledReports\API.sendReport_29...
sh: 1: -t: not found
INFO [2018-11-01 15:42:53] 79180  Scheduler: finished. Time elapsed: 22.156s

Anyone have any idea what this might be about?

Thanks,
Markus

Matomo version: 3.6.1
MySQL version: 10.0.36-MariaDB-0+deb8u1
PHP version: 7.2.11

1 Like
2

Hi,

Many thanks for your report. I feel like you are very close to helping us solve a larger bug.
I have been getting many reports of people not able to to send reports via cron, but had no way to reproduce it.

Therefore I have a few questions:

  • Can you check the PHP error log to find out which line in which PHP file is exactly causing the issue?
  • Can you provide more details to which operating system your server is using?
  • Can you report back what the response of df -t nfs on the shell would be?
    • this is the only place in Matomo I could find a -t parameter.
3

Hi Lukas,

Thanks for your answer.

Can you provide more details to which operating system your server is using?

Debian 8.11, udates are all installed.

We build the corresponding PHP packages ourselves, since Debian 8 does not contain PHP 7.2.

# statistics @ *** in ~ disk:28% [12:52:04] 
$ php -v
PHP 7.2.11 (cli) (built: Oct 11 2018 02:26:59) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
    with Zend OPcache v7.2.11, Copyright (c) 1999-2018, by Zend Technologies
    with Xdebug v2.6.1, Copyright (c) 2002-2018, by Derick Rethans

# root @ *** in ~ disk:28% [12:54:01] C:1
$ apt search php72
Sorting... Done
Full Text Search... Done
ops-php72/unknown,now 7.2.11-ops-8123 amd64 [installed]

Can you report back what the response of df -t nfs on the shell would be?

We don’t have an nfs partition, so the output is:

# statistics @ *** in ~ disk:28% [12:49:13] 
$ df -t nfs
df: no file systems processed

Output without nfs as parameter:

# statistics @ *** in ~ disk:28% [12:49:19] C:1
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1       576G  125G  428G  23% /
udev             10M     0   10M   0% /dev
tmpfs           1.2G  236K  1.2G   1% /run
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           2.4G     0  2.4G   0% /run/shm

Can you check the PHP error log to find out which line in which PHP file is exactly causing the issue?

I’ll see if I can find anything.

4

Can you check the PHP error log to find out which line in which PHP file is exactly causing the issue?

These are the only two entries in the log php-error.log from today.

[02-Nov-2018 00:57:57 UTC] Error in Matomo (tracker): Error query: SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction In query: UPDATE piwik_log_visit SET visit_last_action_time = ?, visit_total_time = ? WHERE idsite = ? AND idvisit = ? Parameters: array (   0 => '2018-11-02 00:57:06',   1 => 1000,   2 => 1,   3 => '68967688', )
[02-Nov-2018 00:57:57 UTC] Error in Matomo (tracker): Error query: SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction In query: UPDATE piwik_log_visit SET visit_last_action_time = ?, visit_total_time = ? WHERE idsite = ? AND idvisit = ? Parameters: array (   0 => '2018-11-02 00:57:06',   1 => 1000,   2 => 56,   3 => '68967689', )
5

this is the only place in Matomo I could find a -t parameter.

Found another place, but from a plugin:

# statistics @ *** in ~/www/plugins on git:5444afd x disk:28% [13:35:43] 
$ grep -ri "\ \-t" | grep -v ".js" | grep -v Binary | grep -v ".yml" | grep -v ".txt"
SearchEngineKeywordsPerformance/vendor/phpseclib/phpseclib/phpseclib/Net/SCP.php:        if (!$this->ssh->exec('scp -t ' . escapeshellarg($remote_file), false)) { // -t = to

Scp is installed:

# statistics @ *** in ~/www/plugins on git:5444afd x disk:28% [13:35:45] 
$ scp
usage: scp [-12346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file]
           [-l limit] [-o ssh_option] [-P port] [-S program]
           [[user@]host1:]file1 ... [[user@]host2:]file2
6

Hi,

Many thanks for the many details. I forwarded them to the team, as I am running out of ideas.

So this i kind of a dead end, as it is working for me (and many other people) using debian without nfs.

I saw this one too, but I really can’t believe a part of a library that probably isn’t even used causes this issue.

Only thing I can think of is temporarily setting log_level = DEBUG and then checking the Matomo log. (tmp/logs/piwik.log)

7

Thanks to @SteveG for pointing me in the right direction:

http://axiac.ro/blog/sh-1-t-not-found/

It seems that for whatever reason PHP doesn’t know the path to sendmail and therefore executes -t -i instead of sendmail -t -i. So the error lies even outside of Matomo, but in PHP itself.

Chances are that php-cli has a different php.ini which is missing the correct configuration.

8

That is a pretty big find!

Btw, it should look something like this:

[14:02 root@drupal-cms ~] > php -i | grep sendmail
sendmail_from => no value => no value
sendmail_path => /usr/sbin/sendmail -t -i  => /usr/sbin/sendmail -t -i 
Path to sendmail => /usr/sbin/sendmail -t -i

Any possibility to sneak in a ini_set() into the code for this?

9

sendmail_path is set:

# statistics @ eda01 in ~ disk:28% [14:09:37] 
$ /opt/php/php72/bin/php -i | grep sendmail
sendmail_from => no value => no value
sendmail_path => /usr/sbin/sendmail -t -i => /usr/sbin/sendmail -t -i
Path to sendmail => /usr/sbin/sendmail -t -i

# statistics @ eda01 in ~ disk:28% [14:09:39] 
$ /usr/sbin/sendmail --version             
msmtp version 1.4.32
Platform: x86_64-pc-linux-gnu
TLS/SSL library: GnuTLS
Authentication library: GNU SASL
Supported authentication methods:
plain scram-sha-1 cram-md5 external digest-md5 login ntlm 
IDN support: enabled
NLS: enabled, LOCALEDIR is /usr/share/locale
Keyring support: none
System configuration file name: /etc/msmtprc
User configuration file name: /home/statistics/.msmtprc

Copyright (C) 2014 Martin Lambers and others.
This is free software.  You may redistribute copies of it under the terms of
the GNU General Public License <http://www.gnu.org/licenses/gpl.html>.
There is NO WARRANTY, to the extent permitted by law.

But we use msmtp instead of sendmail, which has always been compatible.

10

Then make sure that the Matomo cronjob is using /opt/php/php72/bin/php.

11

Cronjob use this script:

#!/bin/sh
PIWIK_HOST="https://$(grep trusted_host ~/www/config/config.ini.php | head -n 1 | awk '{print $3}' | sed -e 's/"//g')/"
cd ~/www/ \
&& /usr/bin/pgrep -f core:archive > /dev/null \
|| /usr/bin/pgrep mysqldump > /dev/null \
|| /opt/php/php72/bin/php --define memory_limit=6G -f ~/www/console core:archive --url=${PIWIK_HOST} >> ~/log/piwik-archive.log 2>&1
12

A simple e-mail script also works.

<?php 
    ini_set( 'display_errors', 1 );
    error_reporting( E_ALL );
    $from = "test@***.ch";
    $to = "markus@***.ch";
    $subject = "PHP Mail Test script";
    $message = "This is a test to check the PHP Mail functionality";
    $headers = "From:" . $from;
    mail($to,$subject,$message, $headers);
    echo "Test email sent";
?>

# statistics @ *** in ~/www on git:5444afd x disk:28% [14:57:42] 
$ /opt/php/php72/bin/php mail.php 
Test email sent
13

Another Question: Now temporarily downgraded to php5. So everything works as usual. However, the October monthly reports were not sent. How can I repeat the e-mail transmission? Can I somehow mark the reports as “not sent” in the database?

I already tried this here: How do I tell Matomo to invalidate the past historical reports so they can be re-processed from the logs? - Analytics Platform - Matomo But it doesn’t affect the email reports.