Cron job failing - no error message to be found

Support & Bugs
1

Ever since this morning, our archive cron job has been failing with the following error:

$ /usr/bin/php5 /srv/www/webstats.zedcore.com/web/console core:archive --url=https://webstats.zedcore.com/

[Exception]
1 total errors during this script execution, please investigate and try and fix these errors.

core:archive [–url=“…”] [–force-all-websites] [–force-all-periods[=“…”]] [–force-timeout-for-periods[=“…”]] [–skip-idsites[=“…”]] [–force-idsites[=“…”]] [–force-periods[=“…”]] [–force-date-last-n=“…”] [–force-date-range[=“…”]] [–concurrent-requests-per-website[=“…”]] [–disable-scheduled-tasks] [–accept-invalid-ssl-certificate] [–xhprof]

There are no errors logged to the syslog (which is PHP’s error_log setting) or the Apache error logs. I’ve even tried adding

log_writers = “file”
to the config.ini.php file and nothing is logged there either. Any ideas how I can fix this error? Currently running Piwik 2.11.1.

2

Could you run console core:archive with the “-v” option (verbose). It will show more log information.


$ /usr/bin/php5 /srv/www/webstats.zedcore.com/web/console core:archive --url=https://webstats.zedcore.com/  -v

I don’t understand why no log is showing here because clearly it should, even without the verbose option… Or maybe the error happened so early in the process that nothing got logged prior to that.

FYI nothing is logged to file when running with the console, the reason for that is simply that all log gets sent to the output, so putting it into the file would be redundant and just pollute the log file with information you already have.

Well in your case you don’t have that information :slight_smile: but that’s not the normal behavior…

3

OK, running with -v gives the following exception:


Exception trace:
 () at /srv/www/webstats.zedcore.com/web/core/CronArchive.php:406
 Piwik\CronArchive->logFatalError() at /srv/www/webstats.zedcore.com/web/plugins/CoreConsole/Commands/CoreArchiver.php:43
 Piwik\Plugins\CoreConsole\Commands\CoreArchiver->execute() at /srv/www/webstats.zedcore.com/web/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:253
 Symfony\Component\Console\Command\Command->run() at /srv/www/webstats.zedcore.com/web/vendor/symfony/console/Symfony/Component/Console/Application.php:874
 Symfony\Component\Console\Application->doRunCommand() at /srv/www/webstats.zedcore.com/web/vendor/symfony/console/Symfony/Component/Console/Application.php:195
 Symfony\Component\Console\Application->doRun() at n/a:n/a
 call_user_func() at /srv/www/webstats.zedcore.com/web/core/Console.php:58
 Piwik\Console->Piwik\{closure}() at /srv/www/webstats.zedcore.com/web/core/Access.php:454
 Piwik\Access::doAsSuperUser() at /srv/www/webstats.zedcore.com/web/core/Console.php:59
 Piwik\Console->doRun() at /srv/www/webstats.zedcore.com/web/vendor/symfony/console/Symfony/Component/Console/Application.php:126
 Symfony\Component\Console\Application->run() at /srv/www/webstats.zedcore.com/web/console:27

Does that help pinpoint the issue at all?

4

Nope :frowning:

You really have no other output before this? Are you running the command manually in a terminal or is it a cron job?

If you really can’t solve it you could try editing the core/CronArchive.php file at line 828 (see the line on GitHub: https://github.com/piwik/piwik/blob/no-data-welcome/core/CronArchive.php#L828-828). You could change the function to this:


    public function log($m)
    {
        $this->output .= $m . "\n";
        Log::info($m);
        echo $m . "\n";
    }

That way at least the messages should show on the output and it will help you debug.

5

OK, changing the log method got the error to appear:


SUMMARY OF ERRORS
Error: Got invalid response from API request: https://webstats.zedcore.com/index.php?module=API&method=CoreAdminHome.runScheduledTasks&format=csv&convertToUnicode=0&token_auth=147c2d93dd00d72a08b476606f26a779&trigger=archivephp. Response was 'task,output Piwik\Plugins\UserCountry\GeoIPAutoUpdater.update,ERROR: GeoIPAutoUpdater: failed to download 'http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz' to '/srv/www/webstats.zedcore.com/web/misc/GeoIPCity.dat.dat.gz': Error while creating the file: /srv/www/webstats.zedcore.com/web/misc/GeoIPCity.dat.dat.gz'

It appears that the web server couldn’t write the GeoIP files to the “misc” directory, so I’ve changed the permissions on it to allow that (after ensuring PHP was disabled in that dir). After changing the permissions the cron job runs without an error. Interestingly, the “misc” directory didn’t come up in the list of directories requiring write access when I ran the system test from the admin area.

Still unsure why the logger isn’t outputting anything. I had the same output (or lack of it) whether it was running from cron or the command line.

6

Ah I didn’t think of that sooner but now the logger is in a plugin (just for the sake of organizing the code better). But maybe when you updated the plugin’s file were not installed (random updater failures happen sometimes), or maybe for whatever reason the plugin is not enabled…

Try running this command:


./console plugin:activate Monolog

You should see “The plugin is already activated”. If you see something else, that should tell you if it wasn’t activated (that means a 2.11 upgrade was not executed correctly) or if the plugin wasn’t installed correctly during the update.

7

It looks like the plugin wasn’t activated, enabling the Monolog plugin seems to have fixed the log output. There’s a possibility that I forgot to set the permissions on config.ini.php when doing the upgrade to 2.11, so that might have been the cause. I’ll make sure I don’t forget that in future updates…

Thanks for your help!

8

I’m happy to finally know the source of the problem :slight_smile: