Cronjob working in console, but not sending to Piwik


#1

Hi,
When I set the crontab exactly as said in the doc. With the right path, the right time 5 * * * *, and over www-data, then the data is not processed into Piwik interface.
When I check the server over the terminal with the su “…” command, then it puts out to me the data.
After this test, the data shows up in piwik suddenly.
When the cronjob runs again at 00:05 in the night, there are no data in piwik interface to see anymore, for the last day again.
Why this?

Cheers.
Andre


(Matthieu Aubry) #2

Maybe you are using a user crontab, then you should put the user first as explained in: How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo


#3

No, sorry.

First I have done like this in parallels confixx admin area (for the user):
5 * * * * www-data /pathtopiwik/misc/cron/archive.sh > /dev/null

Checked it in the console, if it is working. XML output happened.
Then after 00:05 the archive script did run, but the data wasnt shown for the day in piwik interface, SINCE the day I set up the crontab.

Then when I checked in the console again, all data was shown in piwik interface. Also, since the crontab was set up (16th)

Now yesterday, I deleted the crontab in confixx admin panel and set it up over the console.


# m h  dom mon dow   command
MAILTO="my@email.com"
5 * * * * www-data /var/www/web0/html/piwik/misc/cron/archive.sh > /dev/null

I will see if this changes anything.

But what is the relation to the piwik help doc?
In the doc they write, the archive script does run every hour, when the crontab is set this way, but 5 * * * * means at 5 minutes after midnight.

Is archive script meaning archiving which cant be seen?

What is the relation of archive to the 3.600 sec intervall and “not trigger when browser visits”?

I did read that text about twenty times now, and still dont understand what they want to tell me there. I have a 2.000 visitors a day growing, and when piwik is triggering visits directly, my site stops loading often.

Any help from the team maybe??

Cheers.
Andre


#4

Now I have set the crontab in console to


0 * * * * www-data /var/www/web0/html/piwik/misc/cron/archive.sh > /dev/null

(this should mean every hour, right?? )

and
3.600 sec intervall and "not trigger when browser visits"
it still doesnt write into piwik interface until I check it in console with
su command

What is it?

Thanks.
Andre


(Matthieu Aubry) #5

5 * * * * means every hour at 5 past the hour/

It sounds like your crontab works when you run it directly, but not when it is supposed to run automatically? please ask your web host for info as it is not a piwik issue


#6

I have the exact same replication as “webass”. Cron runs but does not updates. Data shows suddenly (probably midnight). Using CPanel.


#7

[quote=matt]
5 * * * * means every hour at 5 past the hour/

It sounds like your crontab works when you run it directly, but not when it is supposed to run automatically? please ask your web host for info as it is not a piwik issue[/quote]
And to conclude,


0 * * * * www-data /var/www/web0/html/piwik/misc/cron/archive.sh > /dev/null

means it runs every hour at the beginning of the hour.
But in the piwik doc they write 5 * * * * will have it run once a day, 5 minutes after midnight. For people setting up a crontab the first time, this is badly irritating.

So far so good.
My hoster will tell me, "Hey, this is your VPS. Propably a problem of the piwik script."
But I will try parallely.

Matt, in what relation comes the 3.600 sec intervall together with the crontab?
I mean, the crontab is for archiving, and the second intervall is for processing data. What is the difference?
Is there a difference or just different wording, for the same?

Thanks in advance.


#8

I guess the Cron is only for “today” stats. Either on demand when the page is requested on through Cron.

The Daily, Weekly, Monthly and Yearly stats are generated only in the night. That’s “the data shows up in piwik suddenly”.

Here is from Piwik FAQs:

How long does the Piwik archiving process take for high traffic websites - or hundreds of websites?

Every night, if you setup auto-archiving, Piwik will run through the logs and compile all your reports for each day, week, month and year…

Right?


#9

Anyway.
Since I have set the processing of data each time a browser client is making data to off, my site is running incredibly faster.
I would need to have that cronjob running correctly.
My cronjob is set to run hourly. It is exactly the comman which is used in the piwik doc, I just changed it to run hourly.

It still doens t give the data over to the piwik installation, although all files are writable.
I dont understand this.
There hasnt been any reports or archiving at all.

I know for sure, because my adsense shows what has happened for today.

Any help please??? Maybe from the team?


(Thomas Seifert) #10

Is www-data the user your webserver is running under?
Where from do you conclude that the cron is being executed?
Try removing the “> /dev/null” part to get hopefully some output (and maybe errors) from the script.


#11

Hi,

Is www-data the user your webserver is running under?

when I run top in terminal I get an answer fromscreen:


1234 www-data  16   0  300m  79m  62m S   19  3.9   1:24.79 apache2

so I guess, that www-data is the user apache runs under.

Where from do you conclude that the cron is being executed?

I think the script is run, because when I test the command with “sh” it gives me xml output in console, for day/month/year
as I wrote earlier.

Try removing the “> /dev/null” part to get hopefully some output (and maybe errors) from the script.

I get some output, but only in the terminal, not in Piwik. So should I try?

For conclusion again:
the cronjob is set to 0 * * * * www-data command
I dont get an output in piwik interface, after logging in. at no time. It doesnt show results, since the crontab is set active.
Only when I test the cronjob in the terminal, the PIWIK interface is filled with data.
After the cronjob runs as planned, the interface in PIWIK is emptied again, back to the first day the cronjob was set (two days back from today). There are no files.

When I test the crontab with


sh /var/www/web0/html/piwik/misc/cron/archive.sh

or as described in the piwik doc, THEN the PIWIK interface is fille dwith data again and will be deleted again, when the cronjob runs automatically the next time (next full hour).

What can this be?
In the meantime I will try to left the /dev/null out, because it will run in 9 minutes. :wink:
Thanks so far.
Andre


#12

In the meantime I will try to left the /dev/null out, because it will run in 9 minutes.

I deleted it, but didnt recieve an error email.

Now, I go to terminal and run it:


sh /var/www/web0/html/piwik/misc/cron/archive.sh

and I get the XML output without any error shown.
Mmh.
Do I have to live with it?
Set the crontab off and let the archive script only run in terminal when I need it?

And then, I still dont let PIWIK run the archive and the process everytime, the browser sends data.
This way, I could have my data available all the time, and re-new it when I need, and ALSO have enough performance on page, right?


(Matthieu Aubry) #13

his was a doc issue, now fixed
http://piwik.org/docs/setup-auto-archiving/

So far so good.
My hoster will tell me, "Hey, this is your VPS. Propably a problem of the piwik script."
But I will try parallely.

Matt, in what relation comes the 3.600 sec intervall together with the crontab?
I mean, the crontab is for archiving, and the second intervall is for processing data. What is the difference?
Is there a difference or just different wording, for the same?

Thanks in advance.

If you run the crontab several times in the 3600 s window, it will not do any work, just exit quickly (after displaying the XML).

Basically if you set the crontab every hour, the timeout should also be set to 3600.


#14

HI,
I now deleted the cronjob again.

I only archive the data per console, when I like over the shell and everything is nice.

This takes the load from the page to a minimum, because it only loads, when I want to look at it, after I do run the command by hand.
Because I did exactly set the cronjob as said, and it didnt do, as I desired it to.

Thanks so far.


#15

Hi there,

I’m new here.

I installed Piwik in a high traffic website, 5k visits day / 15k page views day.

So, I configured a cronjob through the cpanel like you recomends, but it doesnt work. Im with the same problem like “Webass”.

In the console it works great, cronjob clean data in control panel.

Script has execution permissions, so this isnt the problem.

Any help?


(Matthieu Aubry) #16

Have you tried both formats (user VS system crontab formats?) see How to Set up Auto-Archiving of Your Reports - Analytics Platform - Matomo


(vipsoft) #17

As Thomas suggested, remove the redirect to >/dev/null to assist in the troubleshooting.

Four things to check are:

  1. the cron script runs as the same user that runs the apache+mod_php process or under php-cgi+php_suexec
  2. the cron user has the php-cli executable in its path; this is typically configured separately from your shell PATH
  3. the shell script has the execute bit set for the user
  4. the cron user has the right umask