Update from 3.3.0 to 3.4.0 not offered in Matomo

codifex · April 8, 2018, 6:37pm

We always updated Matomo to the newest version using its built-in update options. The update to the newest version was offered inside Matomo/Piwik.

However, Matomo 3.4.0 has been released already some days ago and the update is not offered inside Matomo. In addition, we also did not reveive an e-mail informing us about the new release, which has always been the case for new versions in the past.

When checking for updates inside Matomo 3.3.0, the check returns that the current version of Matomo is already installed.

This problem occurs on multiple independent installs of Matomo.

Any help on how to update automatically is highly appreciated.

fdellwing · April 9, 2018, 6:54am

Try reloading the admin page without cache, this helps my every time.

Lukas · April 9, 2018, 7:05am

If you want to you can also clear the update cache like that:

DELETE FROM `piwik_option` WHERE `piwik_option`.`option_name` = 'UpdateCheck_LastTimeChecked'

codifex · April 10, 2018, 8:51pm

Unfortunately both solutions did not solve the problem, the update check still shows that I am using the latest version although the version is 3.3.0. I also deleted the contents of the temporary directory, but it had no effect.

I know that I can replace the files on the server with the current release, but is there any way to fix the problem or to invoke the update routine via Matomo’s interface?

Any suggestions are appreciated. The problem affects multiple (all of my) installs of Matomo which are running on different systems.

fdellwing · April 11, 2018, 8:08am

Please open the DevTools of your browser and navigate to the network tab. Than press the “search for updates” button in Matomo. Tell us what happens in the devtools

codifex · April 11, 2018, 8:36am

According to the dev tools of the current version of Firefox:

When clicking the “search for updates” button, a HTTP POST request to “…/index.php?date=today&module=CoreHome&action=checkForUpdates&idSite=1&period=day” is made that gets answered with HTTP status code 200 OK.

Response header:

HTTP/1.1 200 OK
Cache-Control: no-store, must-revalidate
Connection: close
Content-Encoding: gzip
Content-Length: 22
Content-Security-Policy: default-src 'self'; img-src 'self' https: data:; script-src 'self' 'unsafe-inline' 'unsafe-eval' https: *.facebook.com; font-src 'self' data:; frame-src https: *.facebook.com; child-src https: *.facebook.com; style-src 'self' 'unsafe-inline'
Content-Type: text/html; charset=utf-8
Date: ...
Expires: 
Pragma: 
Server: Apache
Strict-Transport-Security: max-age=15768000
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-Frame-Options: sameorigin
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block

The response content is almost empty according to dev tools (only 2 bytes which produce new lines in the preview).

I already tried removing the various security headers, but the result has been the same.

I enabled full error reporting in PHP, but no error/warning messages are shown.

No error messages in the web server’s log files.

fdellwing · April 11, 2018, 9:22am

If there are no PHP errors, the problem should be something in this functions:

    public static function check($force = false, $interval = null)
    {
        if (!SettingsPiwik::isAutoUpdateEnabled()) {
            return;
        }

        if ($interval === null) {
            $interval = self::CHECK_INTERVAL;
        }

        $lastTimeChecked = Option::get(self::LAST_TIME_CHECKED);
        if ($force
            || $lastTimeChecked === false
            || time() - $interval > $lastTimeChecked
        ) {
            // set the time checked first, so that parallel Piwik requests don't all trigger the http requests
            Option::set(self::LAST_TIME_CHECKED, time(), $autoLoad = 1);

            $latestVersion = self::getLatestAvailableVersionNumber();
            $latestVersion = trim((string) $latestVersion);
            if (!preg_match('~^[0-9][0-9a-zA-Z_.-]*$~D', $latestVersion)) {
                $latestVersion = '';
            }

            Option::set(self::LATEST_VERSION, $latestVersion);
        }
    }

    private static function getLatestAvailableVersionNumber()
    {
        $releaseChannels = StaticContainer::get('\Piwik\Plugin\ReleaseChannels');
        $channel = $releaseChannels->getActiveReleaseChannel();
        $url = $channel->getUrlToCheckForLatestAvailableVersion();

        try {
            $latestVersion = Http::sendHttpRequest($url, self::SOCKET_TIMEOUT);
        } catch (\Exception $e) {
            // e.g., disable_functions = fsockopen; allow_url_open = Off
            $latestVersion = '';
        }

        return $latestVersion;
    }

So, some first easy questions after looking at them:

You have auto update enabled?
You are on the correct release channel?

codifex · April 11, 2018, 9:46am

Thanks a lot for your effort.

According to the settings I am on the recommended stable release channel.

I just saw that the option regarding notification e-mails was set to “no”, although I received these e-mails in the past. I now set it to “yes”.

fdellwing · April 11, 2018, 9:56am

Ok, please run the following commands against the config dir in Matomo.

grep -R 'show_update_notification_to_superusers_only'
grep -R 'enable_internet_features'
grep -R 'enable_auto_update'
grep -R 'enable_update_communication'

codifex · April 11, 2018, 10:49am

Here is the result:

$ grep -R 'show_update_notification_to_superusers_only'              
global.ini.php:show_update_notification_to_superusers_only = 0                                      
$ grep -R 'enable_internet_features'                                 
global.ini.php:enable_internet_features = 1                                                         
$ grep -R 'enable_auto_update'                                       
global.ini.php:enable_auto_update = 1                                                               
$ grep -R 'enable_update_communication'                              
global.ini.php:enable_update_communication = 1

fdellwing · April 11, 2018, 10:56am

Ok, I have no further idea than. Maybe @matthieu or @SteveG have any?

Lukas · April 11, 2018, 3:26pm

Hi @codifex,

Last idea before I am also running out of ideas.

Can you apply the following change to the core/UpdateCheck.php file?

 diff --git a/core/UpdateCheck.php b/core/UpdateCheck.php
index 403fa90b36..281b3d7dfc 100644
--- a/core/UpdateCheck.php
+++ b/core/UpdateCheck.php
@@ -66,7 +66,7 @@ class UpdateCheck
         $releaseChannels = StaticContainer::get('\Piwik\Plugin\ReleaseChannels');
         $channel = $releaseChannels->getActiveReleaseChannel();
         $url = $channel->getUrlToCheckForLatestAvailableVersion();
+        var_dump($url);
         try {
             $latestVersion = Http::sendHttpRequest($url, self::SOCKET_TIMEOUT);
         } catch (\Exception $e) {

Now the update check (the POST with &action=checkForUpdates) will print the URL it checks on the Matomo API to display if there is a new error.

For me the URL is http://api.matomo.org/1.0/getLatestVersion/?piwik_version=3.4.0&php_version=7.1.15-1&mysql_version=5.5.5&release_channel=latest&url=http%3A%2F%2Flocalhost%2Fpiwik%2Findex.php&trigger=CoreHome&timezone=Europe%2FVienna and it properly returns 3.4.0.

codifex · April 12, 2018, 8:01am

Hello, for me it prints (domain removed):

http://api.matomo.org/1.0/getLatestVersion/?piwik_version=3.3.0&php_version=7.2.4&mysql_version=5.5.59&release_channel=latest_stable&url=...index.php&trigger=CoreHome&timezone=Europe%2FVienna

Calling the URI above (including the domain, as printed by var_dump) returns 3.4.0.

However, the problem still persists. Any ideas are very welcome!

codifex · April 13, 2018, 3:02pm

I modified the function in “UpdateCheck.php” as follows:

private static function getLatestAvailableVersionNumber()
{
    $releaseChannels = StaticContainer::get('\Piwik\Plugin\ReleaseChannels');
    $channel = $releaseChannels->getActiveReleaseChannel();
    $url = $channel->getUrlToCheckForLatestAvailableVersion();

    try {
        $latestVersion = Http::sendHttpRequest($url, self::SOCKET_TIMEOUT);
    } catch (\Exception $e) {
        // e.g., disable_functions = fsockopen; allow_url_open = Off
        $latestVersion = '';
    }
echo 'TEST: Latest Version |' . $latestVersion . '|';
    return $latestVersion;
}

The line I added reveals that the $latestVersion variable contains the HTML code of a SQUID error page, text content:

ERROR
The requested URL could not be retrieved

The following error was encountered while trying to retrieve the URL: http://api.matomo.org/1.0/getLatestVersion/?

    Access Denied.

Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.

Your cache administrator is root.

Generated Fri, 13 Apr 2018 14:50:12 GMT by [proxy server] (squid/3.5.20)

It seems that Http::sendHttpRequest does not throw an error because the SQUID proxy server returns the SQUID error page’s content. I will contact the server administrator as it seems that for some unknown reason the HTTP request to the Matomo API gets blocked by SQUID.

If these findings are correct, I suggest to extend the function posted above to check the validity of the content (version number expected) received by the HTTP request.

Lukas · April 13, 2018, 3:22pm

Many thanks for finding out the reason.

You are right that falling back to doing nothing isn’t ideal, I have opened a github issue:

github.com/matomo-org/matomo

Matomo should throw an error if the request to api.matomo.org fails

opened 03:21PM - 13 Apr 18 UTC

Findus23

c: Usability

After long debugging of an issue where no update was found, it turned out that a… proxy was blocking the request. https://forum.matomo.org/t/update-from-3-3-0-to-3-4-0-not-offered-in-matomo/27786 In my opinion the updater should not act as if no update is available if the request fails. On the contrary always showing warnings may be annoying in case api.matomo.org is down once. Maybe a warning could only be shown after repeated retries.

codifex · April 13, 2018, 4:03pm

Thank you for opening the github issue!

I have contacted the server admin the configure an exception for api.matomo.org.

To my knowledge the updater will download the current release from the matomo.org server via HTTP. It would be great if you could provide the domain name + subdomain so I can forward them to the server admin too.

Lukas · April 13, 2018, 4:05pm

While I haven’t checked the updater, all Matomo versions are here: https://builds.matomo.org/

So I guess that also the updater is fetching them from there.

codifex · April 16, 2018, 8:21am

The server admin excluded the Matomo domain from SQUID and as expected the problem has gone.

ACiDC0re · May 24, 2018, 2:30pm

Hi,
sorry for Hijacking

But I have a similar problem.

I edited the UpdateCheck.php like codifex.

private static function getLatestAvailableVersionNumber()
    {
        $releaseChannels = StaticContainer::get('\Piwik\Plugin\ReleaseChannels');
        $channel = $releaseChannels->getActiveReleaseChannel();
        $url = $channel->getUrlToCheckForLatestAvailableVersion();

        try {
            $latestVersion = Http::sendHttpRequest($url, self::SOCKET_TIMEOUT);
        } catch (\Exception $e) {
            // e.g., disable_functions = fsockopen; allow_url_open = Off
            $latestVersion = '';
        }
echo 'TEST: Latest Version |' . $latestVersion . '|';
        return $latestVersion;
    }

And it returns an empty string

I can ping api.matomo.org and curl returns:

curl "http://api.matomo.org/1.0/getLatestVersion/?piwik_version=3.3.0&php_version=7.1.16&mysql_version=5.5.5&release_channel=latest_stable&url=https%3A%2F%2Fstats.domain.com%2Findex.php&trigger=CoreHome&timezone=Europe%2FBerlin"
3.5.0